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FOREWORD 


This  document  summarizes  the  procedures  required  to  obtain  ANSI  Z-540  Radar 
Cross  Section  (RCS)  range  certification  as  adopted  by  the  Range  Commanders  Council 
Signature  Measurement  Standards  Group  (RCC/  SMSG).  The  RCS  certification 
demonstration  program  verified  the  feasibility  and  utility  of  RCS  range  certification  and  is 
documented  in  a  separate  volume.  This  specific  report  succinctly  provides  a  quick  "step-by- 
step"  reference  guide  so  that  any  government  or  RCS  range  can  obtain  RCS  certification 
through  the  RCC/SMSG,  thereby  allowing  a  company  or  range  to  perform  and  deliver  RCS 
data  products  to  DoD. 

Organized  around  International  Standards  Organization  (ISO)  Guide  25  and  its  US 
equivalent  standard  (ANSI  Z-540),  the  RCS  certification  process  is  designed  to  establish  a 
technical  and  procedural  quality  system  for  any  arbitrary  RCS  measurement  facility.  Z-540 
is  adopted  to  standardize  the  documentation,  processes,  and  procedures  used  by  all  DoD  and 
industrial  RCS  measurement  facilities.  Once  a  facility  commits  to  obtaining  range 
certification,  this  report  provides  the  necessary  information  to  guide  any  range  through  the 
process. 


Brian  M.  Kent,  Ph.D. 
Research  Fellow 
Member,  RCC/SMSG(Radar) 


Robert  Williams,  Director 
Signature  Technology  Office 
Air  Force  Research  Laboratory 


/ 


Range  Commanders  Council 

Signature  Measurement  Standards  Group  (Radar  Subgroup) 
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EXECUTIVE  SUMMARY 


In  January  1997,  the  Range  Commanders  Council  Signature  Measurement  Standards 
Group  (RCC/SMSG),  in  coordination  with  the  Range  Commanders  Council  (RCC)  Taskmaster, 
established  RCC/SMSG  Task  #08  titled,  “DoD  RCS  Certification  Feasibility  Demonstration  for 
Static  and  Dynamic  RCS  Measurement  Facilities.”  Referred  to  herein  as  the  “DoD  RCS 
Demonstration  Program,”  the  task  assessed  the  feasibility  of  establishing  common 
documentation  standards  for  describing  the  operation  and  capabilities  of  the  Department  of 
Defense’s  (DoD)  diverse  radar  cross  section  (RCS)  measurement  facilities  located  throughout 
the  United  States.  Through  the  mutual  efforts  of  the  Air  Force  Research  Laboratory  (AFRL),  the 
Naval  Air  Warfare  Center  Aircraft  Division  (NAWC-AD),  and  the  U.S.  Air  Force  46  Test  Group 
(46TG),  in  cooperation  with  the  RCC/SMSG  Radar  Committee,  the  DoD  RCS  Demonstration 
Program  was  entirely  successful  and  should  lay  the  groundwork  for  similar  technical  or 
laboratory  calibration  certification  efforts  throughout  DoD’s  RCS  measurement  community  [1], 

This  report  describes  how  the  American  National  Standards  Institute/National  Calibration 
Standards  Laboratory  ANSI/NCSL  Z-540-1-1994  Standard  [2]  (referred  to  herein  as  Z-540)  is 
applied  to  organize  RCS  range  documentation  for  any  DoD  or  industrial  RCS  measurement 
facility.  This  report  is  a  follow-up  to  the  DoD  RCS  Demonstration  Program,  as  recently 
presented  to  the  Antenna  Measurement  Techniques  Association  (AMTA)  [12]  [13].  In  June 
2000,  the  RCC/SMSG  certified  that  two  facilities,  the  AFRL  Advanced  Compact  Range  (ACR), 
and  the  NAWC-AD  Atlantic  Test  Range  (ATR)  Dynamic  RCS  Range  measurement  facilities 
met  the  Z-540  documentation  standards  established  by  the  DoD  RCS  Demonstration  Program. 
Since  AFRL  plans  to  unilaterally  require  mandatory  Z-540  compliance  for  its  DoD  contractors 
performing  RCS  measurements  after  January  1,  2004  the  RCS  certification  review  process 
described  in  this  report  will  likely  be  the  model  for  industrial  compliance.  After  reviewing  the 
Z-540  standard,  this  report  will  summarize  the  certification  review  process  and  provide 
supplementary  appendices  documenting  the  RCS  certification  process. 


IX 


CHAPTER  1 


INTRODUCTION  TO  THE  ANSI/NCSL  Z-540-1-1994  STANDARD 


1.1  General 

This  chapter  presents  a  brief  overview  of  the  American  National  Standards  Institute/ 
National  Calibration  Standards  Laboratory  ANSI/NCSL  Z-540-1-1994  Standard  [2]  (referred  to 
herein  as  Z-540).  Z-540  offers  a  straightforward  way  to  organize  radar  cross  section  (RCS)  range 
documentation.  We  begin  by  discussing  the  major  points  of  Z-540  and  how  to  organize  a  format- 
universal  RCS  Range  Book.  Since  Z-540  is  the  US  equivalent  of  International  Standard 
Organization  (ISO)  25,  it  is  especially  useful  for  two  reasons:  (1)  it  is  applicable  to  RCS  ranges; 
and  (2)  its  quality  control  requirements  are  consistent  with  the  ISO  9000  series  of  quality  standards. 
Properly  applied,  Z-540  will  greatly  improve  the  quality  and  consistency  of  RCS  measurements 
produced  by  RCS  measurement  facilities  and  reported  to  range  and  DoD  customers. 

The  signature  measurement  community  is  well  aware  of  the  complexities  of  a  modem 
RCS  measurement  range.  These  electromechanical  systems,  regardless  of  range  type,  must 
contain  a  myriad  of  hardware  and  software  in  order  to  operate  the  range  within  its  intended 
design  parameters.  Naturally,  operating  the  range  as  designed  helps  assure  that  the  range 
produces  consistently  high-quality  data.  Often,  the  job  of  keeping  current  range  documentation 
may  seem  overwhelming,  especially  since  there  are  currently  no  standards  or  guidelines  for 
organizing  such  documentation. 

During  the  process  of  examining  various  candidates  for  an  RCS  documentation  standard, 
several  factors  were  considered.  First,  from  a  DoD  perspective,  we  examined  typical  military 
specifications  and  standard  federal  contract  report  deliverables  to  see  if  such  standards  could  be 
exploited  and  customized  for  the  documentation  of  RCS  ranges.  In  light  of  the  new  DoD  5000 
series  of  federal  acquisition  regulations,  we  are  seeing  significant  federal  procurement  and 
acquisition  reform.  In  the  future,  DoD  will  quickly  move  away  from  the  use  of  rigid  military 
specifications  (Mil  Specs)  in  federal  contracts,  in  favor  of  more  universally  available  and 
understood  commercial  grade  standards.  Since  future  DoD  contracts  will  not  rely  exclusively  on 
Mil  Specs,  they  were  ruled  out  as  a  possible  RCS  range  standard  model. 

The  RCC/SMSG  Radar  Committee  also  examined  the  ongoing  work  of  the  Institute  of 
Electrical  and  Electronics  Engineers  Antennas  and  Propagation  Society  (IEEE-APS)  Committee 
on  RCS,  of  which  the  National  Institute  of  Standards  and  Technology  (NIST)  is  an  active 
participant.  While  the  work  of  this  important  committee  was  not  complete  in  1997  (and,  as  of 
the  date  of  this  publication,  still  is  not  complete),  it  is  clear  that  the  IEEE-APS  will  not  focus  on 
range  documentation  standards. 

Finally,  the  RCC/SMSG  Radar  Committee  examined  ongoing  work  in  the  European 
community  (EC),  which  already  mandates  most,  if  not  all,  of  the  ISO  9000/9002  series  of 
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regulations  for  their  commercial  business  transactions.  While  ISO  9000  by  itself  is  not 
applicable  to  RCS  ranges,  the  technical  standard  ISO  25  appeared  to  satisfy  most  of  the 
requirements  for  a  quality-oriented  RCS  range  documentation  system.  We,  therefore,  looked 
into  the  US  equivalent  of  ISO  25,  which  is  ANSI/NCSL  Z-540-1-1994. 

1.2  A  Closer  Look  at  ANSI/NCSL  Z-540-1-1994 

The  ANSI/NCSL  Z-540-1-1994  Standard  [2]  is  available  for  a  nominal  fee  from  the 
National  Calibration  Standards  Laboratory  (NCSL)  (tel.  #  303-440-3339).  The  title  of  the  standard 
is  Calibration  Laboratories  and  Measuring  and  Test  Equipment  -  General  Requirements,  and  it 
provides  a  framework  for  collecting  and  maintaining  information  to  support  a  quality-oriented 
system  of  documentation  for  any  range  that  produces  calibrated  data.  In  essence,  the  standard  is 
very  general  and  could  easily  accommodate  many  types  of  calibrated  scientific  measurements 
(e.g.,  voltage,  lumens,  temperature,  etc.).  Basically  a  13-page  guide,  Z-540  summarizes  the 
general  documentation  requirements  for  any  facility  producing  quality  calibrated  data. 

Z-540  is  organized  into  two  parts.  Part  I  is  preceded  by  a  three-section  introduction  and 
scope,  and  includes  references  and  standard  definitions.  The  definition  section  is  especially 
helpful  in  establishing  a  common  dialog  for  the  remainder  of  the  standard.  The  main  thrust  of 
the  standard  is  Part  I  titled,  “General  Requirements  For  The  Competence  of  Calibration 
Laboratories.”  It  consists  of  13  subsections  outlining  the  basic  standard  and  the  information 
needed  for  a  range  to  document  to  the  standard.  Part  II  of  the  standard  titled,  “Quality  Assurance 
Requirements  for  Measuring  and  Test  Equipment,”  pertains  to  the  control,  usage,  and  calibration 
of  individual  pieces  of  equipment  used  in  producing  calibrated  data.  As  will  be  discussed  later, 
Part  I  is  the  most  applicable  and  appropriate  portion  of  this  standard  when  applied  to  the  specific 
case  of  RCS  ranges.  Part  II  was  not  used  in  any  part  of  the  RCS  certification  efforts. 

1.3  Using  Z-540  to  Establish  an  RCS  Range  Quality  System 

The  central  theme  of  Z-540  is  the  establishment  and  maintenance  of  an  active  and 
ongoing  measurement  quality  assurance  program  (MQAP).  Most  ranges  already  incorporate 
“measurement  assurance”  as  part  of  their  normal  range  operations,  although  this  phrase  is  rarely 
quantified.  However,  it  may  be  helpful  to  briefly  define  the  elements  of  a  MQAP. 

A  typical  MQAP  consists  of  documented  technical  requirements  designed  to  ensure 
repeatable  RCS  performance  out  of  a  given  measurement  range.  The  MQAP  documents 
measurement  procedures,  data  analysis  procedures,  range  uncertainty  and/or  error  estimation 
analysis,  and  data  analysis  procedures,  while  simultaneously  identifying  a  sound  management 
structure  set  up  to  efficiently  operate  and  maintain  the  RCS  range.  It  also  strongly  encourages  an 
active  interrange  comparison  program  (within  reasonable  time  and  range  availability  constraints) 
and  an  ongoing  research  program  to  improve  RCS  data  quality.  At  first  reading,  it  may  appear  that 
standard  Z-540  actually  implies  the  development  of  a  separate  MQAP  in  addition  to  the 
documentation  required  by  the  standard.  Actually,  completion  of  the  documentation  required  by 
Z-540  will,  in  effect,  establish  a  documented  MQAP.  Therefore,  we  now  review  the  elements  of 
Z-540  section  by  section. 
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1.4  A  Sectional  Overview  of  Z-540  -  Part  I 


Z-540  begins  with  a  three-section  “Foreword,”  which  defines  the  scope  of  the  standard 
(Section  1),  important  references  to  the  standards  (Section  2),  and  a  set  of  excellent  baseline 
definitions  (Section  3).  These  all  help  the  reader  define  a  common  frame  of  reference  for 
interpretation  of  the  standard.  After  the  Foreword,  Part  I  (General  Requirements  for  the 
Competence  of  Calibration  Laboratories)  presents  the  details  of  the  standard  itself.  Part  I 
consists  of  13  individual  sections  (Sections  4-16),  which  present  the  ideal  organization  for  a 
range  documentation  set  paralleling  the  sections  of  the  standard.  Let  us  review  each  section  by 
title,  and  provide  some  insight  into  their  purpose.  Later,  we  will  define  how  each  item  is 
evaluated  in  practice. 

Z-540  Section  4  (Organization  and  Management)  briefly  describes  how  to  document  the 
organization  performing  the  calibrations.  Using  an  open-ended  format,  the  RCS  range 
organization  and  personnel  would  be  identified  along  with  their  typical  supervisory  reporting 
chain.  This  section  also  identifies  the  technical  and  quality  leaders  responsible  for  the  respective 
technical  and  quality  aspects  of  range  operations.  (Naturally  in  smaller  RCS  ranges,  these  two 
positions  may  be  held  by  the  same  person.)  Z-540  Section  4  requires  all  range  individuals  (and 
any  alternates)  to  be  specifically  identified.  Also,  Section  4  requires  that  the  RCS  range  be 
organized  and  operated  in  such  a  way  as  to  assure  that  RCS  data  confidence,  independent 
technical  judgment,  and  data  integrity  are  always  maintained. 

Z-540  Section  5  (Quality  System,  Audit,  and  Review)  is  one  of  the  three  major  sections 
of  Z-540  and  sets  the  stage  for  the  remainder  of  the  document.  It  states  that  the  range  will 
establish  and  maintain  an  active  “quality  system”  with  appropriate  internal  audits  and  reviews 
suitably  “appropriate  to  the  type,  range,  and  volume  of  calibration  activities  it  maintains.”  This 
sentence  is  very  important  as  it  distinguishes  Z-540  and  ISO  25  from  the  overall  ISO  9000/9002 
series  in  the  important  phrase  “appropriate  to  the  type,  range,  and  volume  of  calibration 
activities...”.  Subsection  5.2  is  more  specific  and  lays  out  the  elements  that  should  be  included 
in  the  quality  manual.  Some  examples  include  (5.2-d)  procedures  to  control  and  maintain 
documentation;  (5.2-g)  procedures  for  achieving  traceability;  (5.2-h)  scope  of  calibration  and 
verification  activities;  (5.2-k)  procedures  for  handling  (calibration)  devices;  (5.2-1)  references  to 
major  equipment  and  reference  measurement  standards;  (5.2-n)  references  to  quality  assurance 
practices,  including  inter-laboratory  comparisons,  proficiency  testing,  and  internal  quality  control 
practices;  (5.2-o)  procedures  for  corrective  action  when  RCS  data  discrepancies  are  found;  and 
(5.2-q)  procedures  for  dealing  with  customer  complaints.  Although  Sections  5. 3-5. 6  describe  a 
few  additional  details  on  the  audit  and  review  process,  the  creation  of  the  range-appropriate 
“quality  manual”  in  Section  5.2,  as  it  pertains  specifically  to  RCS  measurement  ranges,  is  the 
main  intent  of  standard  Z-540. 

Z-540  Section  6  (Personnel)  describes  how  a  range  defines  and  maintains  the  appropriate 
staff  needed  to  operate  the  RCS  range.  It  also  requires  ranges  to  describe  their  initial  and 
ongoing  training  programs  used  to  maintain  the  technical  competence  of  their  operating  staffs. 
This  information  should  be  easy  to  assemble,  as  most  organizations  routinely  obtain  and  manage 
this  type  of  information. 
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Z-540  Section  7  (Accommodation  and  Environment)  requires  a  range  to  describe  its 
environmental  “accommodation”  and  to  identify  what  environmental  factors  may  affect 
measurement  results.  This  latter  category  may  be  very  short  for  some  indoor  ranges  but  lengthy 
for  outdoor  or  dynamic  measuring  applications.  For  instance,  an  outdoor  static  range  may  wish 
to  record  temperature,  winds,  precipitation,  inversions,  or  other  phenomena,  which  are  known  to 
affect  the  RCS  measurement  data.  When  the  effects  cannot  be  quantified  (in  RCS  error  terms),  it 
is  still  very  important  to  identify  and  record  when  such  conditions  occurred. 

Z-540  Section  8  (Equipment  and  Reference  Materials)  describes  how  equipment, 
maintenance,  and  reference  materials  and  records  are  tracked.  Again,  most  organizations  have 
systems  to  control  and  identify  all  equipment  used  by  a  typical  range.  By  systematically 
organizing  this  information  into  a  “range  appropriate”  configuration  control  document,  one 
makes  a  good  external  record  of  the  measurement  system,  as  well  as  an  excellent  and  useful 
internal  document  valuable  to  the  range  technicians  and  engineers.  Note  that  the  more  complex 
the  range,  the  more  important  the  documentation  becomes. 

Z-540  Section  9  (Measurement  Traceability  and  Calibration)  and  Section  10  (Calibration 
Methods)  together  define  the  information  needed  to  establish  traceability  to  “national, 
international,  or  intrinsic  standards  of  measurement.”  For  RCS  ranges,  this  is  usually 
accomplished  through  indirect  “ratio  or  reciprocity  type  measurements,”  wherein  an  unknown 
target  is  measured,  a  “known”  RCS  “standard”  is  measured,  and  a  ratio  is  computed  with  suitable 
adjustments  using  a  highly  accurate  numeric  RCS  estimate  of  the  “known”  calibration  target. 
Though  every  range  performs  calibrations  in  slightly  different  manners,  the  Z-540  requires  that 
such  procedures  be  rigorously  documented,  tested,  and  verified,  whenever  possible,  with  other 
RCS  ranges.  One  of  the  most  important  aspects  of  these  two  sections  is  contained  in  paragraphs 
10.2  -  10.4,  which  describe  the  need  for  written  procedures  to  execute  appropriate  calibrations. 
Once  again,  having  this  procedural  information  universally  available  to  your  range  staff, 
including  archival  results  (e.g.,  what  the  “correct”  calibration  measurements  should  be)  will 
greatly  improve  the  quality  and  repeatability  of  RCS  calibration  values.  In  addition,  specific 
calibration  processes  selected  for  use  by  a  given  range  must  be  routinely  tested.  Calibration 
procedures  discussed  in  [2,  3]  are  one  way  of  meeting  this  requirement,  though  other 
documented  methods  may  be  used  as  well. 

Z-540  Section  1 1  (Handling  of  Calibration  Items)  is  self  explanatory  and  essentially 
states  the  need  for  maintaining  physical  control  over  calibration  items  used  to  produce  calibrated 
RCS  range  data.  Such  items  should  be  appropriately  stored  and  protected  when  not  being  used. 

In  addition,  it  points  out  the  need  to  uniquely  identify  each  primary  range  calibration  target,  so 
that  there  is  never  confusion  regarding  the  use  of  one  calibration  target  over  another. 

Z-540  Section  12  (Records)  is  also  self  explanatory  and  basically  states  that  the  range 
must  have  a  record  keeping  system  consistent  with  “its  particular  circumstances.”  It  also 
specifically  states  that  the  record  keeping  system  should  be  consistent  with  the  overall  range 
quality  system. 

Z-540  Section  13  (Certificates  and  Reports)  describes  the  elements  that  must  appear  in  a 
“range  certificate,”  otherwise  called  an  RCS  data  or  test  report.  It  emphasizes  that  each  range 
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should  have  a  minimum  subset  of  information  in  the  report,  including  range  identity  (title,  name, 
and  address  of  range);  customer  identity  (name  and  address  of  customer);  description  of  the 
calibration  item  (target);  dates  of  the  calibration  measurements,  process,  calibration  techniques 
used,  and  so  on.  It  is  meant  to  provide  consistency  of  data  reporting  from  a  specific  range,  as 
well  as  to  establish  more  universal  reporting  standards  from  range  to  range. 

Z-540  Sections  14  (Subcontracting  of  Calibration)  and  Section  15  (Outside  Support 
Services  and  Suppliers)  are  both  brief,  but  important.  In  essence,  they  state  that  if  outside 
suppliers  are  used  in  any  part  of  the  process  of  supplying  calibrated  RCS  data,  such  suppliers  are 
subject  to  the  requirements  of  the  Z-540  standard,  if  the  prime  or  supplying  range  conforms  to 
this  standard. 

Finally,  Z-540  Section  16  (Complaints)  assures  that  the  range  has  a  formal,  quality- 
oriented  policy  for  responding  to  customer  complaints  regarding  any  aspect  of  range  operation. 

1.5  Organizing  the  RCS  Range  Book  to  Comply  with  Z-540 

The  following  provides  guidelines  for  the  construction  of  the  RCS  Range  Book  used  to 
document  range  characteristics  in  accordance  with  Z-540.  The  goal  of  these  guidelines  is  to 
maintain  consistency  in  the  layout  of  the  documentation  from  range  to  range.  Appendix  I  of  this 
document  provides  a  detailed  description  of  the  layout  of  a  typical  range  book,  so  only  the 
general  points  will  be  repeated  here. 

The  main  goal  of  the  range  book  is  to  create  and  organize  the  RCS  system  documentation 
into  a  format  independent  of  that  used  by  the  ranges.  However,  within  each  section  of  the  range 
book,  company  or  local  documentation  formatting  is  strongly  encouraged,  whenever  possible.  In 
this  manner,  substantial  costs  for  reformatting  information  to  Z-540  are  avoided  to  the  extent 
possible,  while  at  the  same  time  affording  a  common  document  layout.  The  goal  here  is  to  create 
a  "living  breathing  document"  useful  to  the  range  staff  for  whom  it  was  created.  It  is  the  range 
book  that  is  reviewed  technically  by  a  third  party  and  is  the  basis  for  granting  a  range 
certification.  This  factor  will  be  discussed  later  in  this  report. 

The  range  book  sections  will  closely  parallel  the  Z-540  standard,  with  few  exceptions. 
However,  it  is  emphasized  that  the  importance  of  the  RCS  Range  Book  is  in  its  basic  utility  as  a 
general  reference  standard  for  range  personnel  and  customers  alike.  Thus,  the  utility  of  the  range 
book  will  largely  depend  on  the  currency  and  relevance  of  the  information  in  it  and  the  amount 
of  effort  expended  to  create  it. 

Before  discussing  the  range  book  format,  we  consider  the  issue  of  universal  availability 
and  utility.  Depending  on  the  size  and  type  of  range,  the  documentation  set  may  be  physically 
small  or  large.  For  instance,  we  would  expect  the  documentation  for  a  highly  complex  dynamic 
signature  measurement  range  to  be  substantially  larger  than  a  relatively  simple  indoor  far  field 
range.  However,  any  range  may  want  to  carefully  consider  whether  to  produce  their  range  book 
documentation  in  paper  or  electronic  form.  While  a  conventional  paper  copy  would  certainly 
meet  all  requirements  of  Z-540,  an  electronic  version  (available  on  a  network  server,  internal  or 
external  web  page,  or  a  permanent  CD-ROM)  available  to  every  range  employee  may  be  a  better 
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option  in  the  long  run.  Normally,  if  multiple  hard  copies  of  the  range  book  are  created,  each 
must  be  separately  updated  to  reflect  changes  in  the  range  information  over  a  range's  natural 
lifetime.  In  the  case  of  an  electronic  or  web  page  version,  only  one  copy  (on  the  web  page) 
would  need  to  be  updated  and  redistributed.  Thus,  maintaining  current  documentation 
electronically  should  prove  easier  than  maintaining  paper  copies.  In  either  case,  the  reader  is 
encouraged  to  use  whatever  system  is  available  or  appropriate  for  their  situation. 

The  RCS  Range  Book  (paper  or  electronic)  should  be  the  ready  range  reference  available 
for  use  by  any  range  staff  member.  The  range  book  also  represents  the  official  document  set, 
reviewed  by  a  third  party  RCS  Certification  Review  Committee  as  discussed  later  in  this  report. 
Once  a  range  book  is  completed  and  certified,  it  is  the  responsibility  of  the  range  to  keep  it 
current. 


1.6  RCS  Range  Registrations  and  Third  Party  Certification  Review 

Compilation  of  the  range  book  completes  an  important  phase  in  the  formal 
documentation  of  range  performance.  For  an  RCS  range  to  become  “certified,”  it  must  first 
become  “registered.”  To  become  “registered,”  the  range  forms  its  own  internal  review 
committee  to  check  the  RCS  Range  Book  against  the  published  RCS  evaluation  criteria  provided 
in  Appendix  II  of  this  document.  This  process  is  called  an  internal  “self-review”  audit  or 
“registration”  in  ISO  vernacular.  Once  a  range  book  is  “registered”  internally  and  checked 
against  the  evaluation  criteria,  the  range  management  normally  places  the  appropriate  names  on 
the  opening  page.  Their  joint  signatures  attest  that  management  has  approved  the  overall  quality 
assurance  program  and  the  RCS  Range  Book. 

The  next  part  of  the  certification  process  is  the  independent  third  party  review.  Z-540, 
Section  1.3,  states,  “the  role  of  the  purchaser  in  monitoring  supplier  (range)  compliance  with  the 
requirements  of  this  standard  may  be  fulfilled  by  a  third  party  such  as  a  certification  board.” 
When  DoD  tested  and  demonstrated  a  Z-540-based  RCS  certification  process,  it  relied  on  an  all¬ 
volunteer  RCS  Certification  Review  Committee  formed  by  technical  specialists  drawn  from 
Government,  academia,  and  industry.  In  the  future,  all  range  book  reviewers  will  be  volunteer 
technical  experts  active  in  the  RCS  measurement  business.  Such  a  third  party  review  committee 
“certifies”  that  a  range  complies  with  the  Z-540  standard  as  it  pertains  to  the  specific  RCS 
measurement  range. 

Of  course,  a  third  party  review  committee  can  never  guarantee  that  a  “certified”  range 
will  not  make  errors.  Since  Z-540  is  extremely  process-oriented,  proper  documentation  of 
normal  range  processes  and  procedures  will  produce,  in  our  opinion,  a  quality  system  that  greatly 
reduces  the  occurrence  of  data  problems  related  to  poor  procedural  execution.  The  Z-540 
standard  is  an  excellent  approach  to  organize,  maintain,  and  present  RCS  range  information.  The 
RCS  Range  Book  should  be  very  useful  to  the  range  and  range  customer  alike.  Assuming  that  a 
well-managed  RCS  range  has  much  of  the  required  documentation  in  hand,  complying  with  the 
Z-540  standard  should  be  a  gradual,  transitional  process  as  the  range  book  is  formalized.  If 
omissions  are  found  in  the  RCS  range’s  documentation,  the  process  of  implementing  an  MQAP 
as  part  of  the  Z-540  compliance  process  should  fill  those  omissions,  thereby  improving  the 
quality  and  consistency  of  RCS  range  data  supplied  to  external  customers. 
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CHAPTER  2 


THE  RCS  CERTIFICATION  PROCESS 
STEP-BY-STEP  IMPLEMENTATION 


2.1  General 

This  chapter  provides  a  detailed  overview  of  the  RCS  certification  process.  The  process 
was  thoroughly  tested  by  the  RCC/SMSG,  in  cooperation  with  the  Air  Force  Research 
Laboratory  and  the  Naval  Air  Warfare  Center- Aircraft  Division  (NAWC-AD).  The  RCS  ranges 
that  participated  in  the  original  DoD  RCS  Demonstration  Program  [1]  are  shown  in  Figure  2-1. 
Although  the  original  RCS-based  quality  assurance  documentation  handbook  provided  a  context 
for  the  documentation  standard,  it  provided  few  details  regarding  the  evaluation  criteria  and 
certification  review  processes  needed  to  complete  RCS  certification.  The  remainder  of  this 
report  describes  the  certification  review  process  and  the  evaluation  criteria  used  in  the  RCS 
certification  process  by  the  RCS  Certification  Review  Committee. 


Air  Force  Research  Laboratory 
RVUMS  Outdoor  Static  RCS  Range  Advanced  Indoor  RCS  Range 

(46TG,  Holloman  AFB,  NM)  (Wright-Patterson  AFB,  OH) 


Figure  2-1.  DoD  RCS  Demonstration  Program  Ranges. 
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2.2  Defining  Z-540  RCS  Certification 


To  avoid  repeating  the  background  information  provided  in  Chapter  1,  we  begin  by 
precisely  defining  “RCS  certification.”  From  [1]:  “RCS  certification,  as  used  in  the  RCS 
Certification  Program,  is  hereby  defined  as  having  successfully  completed  a  detailed  review  of 
an  RCS  Range  Book  by  a  peer  review  committee  using  published  evaluation  criteria .” 

The  detailed  review  is  essentially  a  compliance  assessment  that  a  range  has  met  the 
relevant  quality  assurance  documentation  standards  of  Z-540.  In  the  earlier  DoD  RCS 
Demonstration  Program  [1],  “certification”  meant  a  review  committee’s  positive  assessment  of 
compliance  with  the  Z-540  standard  and  an  endorsement  of  the  range’s  measurement  processes. 
RCS  certification  is  NOT  a  rating  system  or  a  specific  review  committee  endorsement  of  day-to- 
day  quantitative  capabilities  (accuracy,  uncertainty,  and  so  forth).  In  the  follow-on  industrial 
RCS  certification  program,  this  definition  and  review  process  will  be  the  same. 

To  better  understand  certification  in  context,  let’s  review  each  major  element  of  the  “RCS 
certification”  definition  above.  We  begin  by  defining  the  RCS  Range  Book.  An  ‘ RCS  Range 
Book ”  (Appendix  I)  is  a  detailed  technical  and  quality  assurance  document  written  in  accordance 
with  the  Z-540  standard  by  the  range  to  be  certified.  The  range  book  thoroughly  describes  all 
aspects  of  the  RCS  measurement  range.  Its  20  distinct  sections  describe  every  major  aspect  of 
range  operations,  procedures,  calibration,  and  equipment.  The  range  book  also  explains  how  the 
range  is  organized  and  operated  on  a  day-to-day  basis.  The  range  book  is  meant  to  be  a  ready 
daily  reference  for  the  range  staff,  as  well  as  a  document  a  potential  customer  could  review  prior 
to  using  the  RCS  facility  for  a  test. 

The  “published  evaluation  criteria  ”  (Appendix  II)  are  a  set  of  approximately  145  specific 
written  evaluation  criteria  that  the  range  book  must  fulfill  in  order  to  become  “RCS  certified.” 
The  evaluation  criteria  provide  the  minimum  information  required  for  each  of  the  20  range  book 
sections.  Since  the  range  knows  the  evaluation  criteria  up  front,  the  criteria  provide  a  definitive 
guide  for  the  range  staff  to  help  prepare  the  range  book  material.  The  evaluation  criteria  are  also 
used  by  the  range  to  “self  review”  or  “register”  the  range  book  by  the  host  organization  prior  to 
certification  review  by  a  third  party. 

The  “peer  review  committee ”  consists  of  three  volunteer  RCS  experts  from  Government, 
industry,  and/or  academia  who  review  the  RCS  Range  Book  for  compliance  with  the  Z-540 
standard  using  the  evaluation  criteria  mentioned  above.  The  volunteer  reviewers  provide 
feedback  to  the  range  on  non-compliant  range  book  sections,  and  work  with  the  range  in  question 
to  revise  the  range  book  to  be  fully  compliant  with  the  standard.  Literally,  the  review  committee 
is  there  to  help  the  range  complete  the  certification  process.  All  range  book  reviewers  must  meet 
minimum  qualifications  set  by  the  RCC/SMSG  Radar  Committee  and  be  approved  by  the  Chair 
of  the  RCC/SMSG. 

In  the  DoD  RCS  Demonstration  Program,  the  Air  Force  Research  Laboratory  (AFRL) 
and  the  Atlantic  Test  Range  facilities  were  solely  reviewed  by  RCS  experts  employed  outside 
DoD  to  avoid  any  perceived  conflict  of  interest.  In  future  reviews,  the  reviewers  may  come  from 
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any  organization  except  the  organization  being  reviewed  for  RCS  certification.  Also,  as  a 
courtesy,  the  names  and  present  employers  of  the  range  book  reviewers  are  provided  to  the 
RCC/SMSG,  who  may  veto  any  potential  reviewer  for  business  or  financial  (real  or  perceived) 
conflicts  of  interest. 

2.3  The  Detailed  Z-540  RCS  Certification  Process 


The  RCS  certification  process  created  by  the  RCC/SMSG  may  seem  complicated  at  first 
glance,  but  it  is  actually  a  manageable,  easy  to  follow  step-by-step  recipe.  Figure  2-2  illustrates 
the  overall  certification  review  process.  The  process  begins  with  the  “Registered”  Phase  when 
an  RCS  range  decides  to  become  “registered”  to  the  Z-540  standard.  This  involves  generating  an 
RCS  Range  Book  (in  paper  or  electronic  format)  that  meets  the  published  evaluation  criteria 
(Appendix  II)  for  RCS  certification.  For  most  ranges,  this  process  can  take  6-24  months 
depending  on  the  complexity  of  the  RCS  range  measurement  system  and  the  range’s  initial  state 
of  documentation.  If  a  range  has  organized  documentation  and  thoroughly  documented 
measurement  and  calibration  procedures  available  at  the  onset,  the  process  can  be  quickly 
completed.  If  these  documents  need  to  be  created  by  the  range  staff,  the  process 


Figure  2-2.  RCS  Certification  Review  Process  and  Timeline . 
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obviously  takes  longer.  Once  the  range  book  draft  is  generated,  the  range  performs  an  internal 
“self  review”  of  the  range  book,  using  the  same  evaluation  criteria  (Appendix  II)  used  by  the 
third  party  review  committee.  After  this  “self  review”  is  successfully  completed,  the  range 
quality  manager  normally  endorses  the  range  book  in  Section  1  (Introduction  and  Endorsements) 
along  with  a  manager  at  least  two  levels  above  the  quality  manager.  Once  reviewed  and  signed 
internally,  a  range  is  considered  “Registered  to  Z-540.”  A  range  must  become  RCS  “registered” 
prior  to  requesting  a  certification  review  from  the  RCC/SMSG  Standards  Committee,  a  practice 
consistent  with  ISO  standard  review  procedures. 

The  next  step  in  the  certification  process  of  Figure  2-2  is  the  Notification  Phase.  The 
range  desiring  certification  contacts  the  RCC/SMSG  Chair  (as  of  the  date  of  this  publication,  Mr. 
Dale  Bradley,  USAF  Arnold  Engineering  Development  Center,  tel.  #  931-454-4242,  e-mail: 
dale.bradley@amold.af.mil)  and  requests  that  an  RCS  Certification  Review  Committee  be 
formed.  Within  30  days  of  being  contacted,  the  Chair  will  assign  a  third  party  team  of  three  RCS 
experts  to  review  the  range  book  in  question.  This  third  party  team  will  consist  of  experts  in 
RCS  recommended  by  the  RCC/SMSG  Radar  Committee.  The  RCC/SMSG  Chair  assigns  the 
members  to  assure  there  is  no  perceived  conflict  of  interest  between  the  reviewers  and  the  range. 
Once  the  review  committee  membership  has  been  named,  the  review  committee  meets  among 
themselves  to  elect  a  chairperson.  By  charter,  the  review  committee  chairperson  is  the  single 
focal  point  between  the  review  committee  and  the  range  under  review  for  the  duration  of  the 
review  process. 

Once  the  review  committee  has  been  formed,  the  review  timeline  officially  starts  and  the 
review  proceeds  into  the  Off-Site  Review  Phase.  The  range  sends  a  copy  of  the  range  book  to 
each  review  committee  member.  The  review  committee  has  three  months  to  review  the  range 
book,  and  grade  it  in  accordance  with  the  evaluation  criteria  in  Appendix  II.  The  chairperson 
gathers  the  individual  grades  and  issues  “composite”  or  committee  grades  for  each  evaluation 
criteria.  At  the  end  of  the  three-month  off-site  review,  the  chairperson  transmits  the  committee’s 
“mid-review”  assessment  to  the  range.  The  mid-review  assessment  will  likely  reveal  areas 
where  the  reviewers  require  additional  documentation,  clarification,  or  improvement  in  some  or 
all  range  book  sections.  Also,  the  review  committee  does  not  initially  grade  about  30%  of  the 
evaluation  criteria,  because  these  criteria  are  evaluated  later  during  the  on-site  staff  interviews. 

If  the  review  committee  identifies  any  marginal  or  unsatisfactory  range  book  sections,  the  review 
committee  must  include  positive  corrective  comments  showing  the  range  how  to  reach 
satisfactory  compliance  in  each  non-compliant  area. 

At  this  point,  the  review  process  moves  into  the  Feedback  and  Update  Phase.  Once  the 
range  receives  the  mid-review  report,  the  range  responds  to  the  feedback  within  14  days  and  is 
given  a  minimum  of  60  days  to  accomplish  range  book  revisions  and  prepare  for  the  on-site 
review.  (More  time  may  be  requested  at  any  time  in  the  process  by  the  range  under  review,  if 
needed.)  The  range  focuses  its  activities  on  correcting  all  high-priority  deficiencies  identified  by 
the  review  committee.  This  is  normally  accomplished  by  adding  written  revisions  to  the  range 
book  to  address  specific  deficiencies.  The  range  also  prepares  for  the  upcoming  review  committee 
on-site  visit.  The  on-site  visit  generally  addresses  three  areas:  (1)  the  summary  presentations  of 
the  overall  range  organization  and  technical  operations  of  the  range,  (2)  deficient  areas  and  the 
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steps  the  range  has  taken  in  the  last  60  days  to  correct  deficiencies,  and  (3)  interviews  with  staff 
and  the  on-site  audit  questions  from  the  evaluation  criteria. 


y  NOTE 


All  on-site  questions  are  known  in  advance.  The  on-site  interviewers  will 
not  deviate  or  add  additional  on-site  questions  from  those  present  in  the 
evaluation  criteria. 


The  final  phase  is  identified  as  the  On-Site  Visit  and  Certification  Phase.  During  this 
period,  the  review  committee  typically  spends  2-4  days  at  the  range  for  an  on-site  visit  of  the 
facility.  In  addition  to  going  over  the  formal  changes  and  revisions  to  the  range  book  described 
previously,  it  provides  an  opportunity  for  the  reviewers  to  get  a  “real  world”  feel  for  the  day-to- 
day  range  operations.  During  this  period,  the  “on-site”  staff  interview  questions  are  asked,  and 
any  final  questions  and  comments  from  the  committee  are  addressed.  At  the  conclusion  of  the 
on-site,  the  review  committee  is  required  to  give  the  range  an  “out-brief’  of  the  new  composite 
evaluation  scores.  Under  most  circumstances,  if  the  range  makes  the  required  revisions,  the 
range  “passes”  and  becomes  RCS  certified  at  this  point.  If,  however,  the  review  committee 
requires  closure  on  a  few  evaluation  criteria,  the  range  may  resubmit  any  final  corrections  to  the 
committee.  Once  the  range  has  met  the  sufficient  number  of  evaluation  criteria  to  “pass”  (see 
Appendix  II),  the  review  committee  issues  a  final  report  to  the  RCC/SMSG  Chair  with  a  copy  to 
the  range.  This  report  provides  the  final  composite  grades  and  concludes  with  a  statement  that 
the  range  has  passed  sufficient  criteria  to  become  “Z-540  RCS  certified.”  The  final  range  book 
Certification  Report  issued  by  the  review  committee  also  becomes  a  permanent  part  of  the  range 
book  (in  Section  5)  at  the  conclusion  of  the  review. 

2.4  Organizing  the  RCS  Range  Book  to  Comply  with  the  Evaluation  Criteria 

The  following  provides  suggested  guidelines  for  the  construction  of  the  RCS  Range  Book 
documentation.  The  goal  is  to  maintain  consistency  in  the  layout  of  the  documentation  from 
range  to  range,  thus  reducing  the  time  needed  to  review  and  assess  compliance  to  the  Z-540 
standard.  Although  Appendix  I  provides  a  detailed  description  of  the  layout  of  a  typical  range 
book,  there  are  several  lessons  learned  in  the  construction  of  the  first  three  DoD  RCS 
Demonstration  Program  range  books  that  are  worth  sharing.  We  hope  range  managers  pay 
attention  to  these  lessons  so  that  ranges  desiring  certification  in  the  future  do  not  repeat  the  same 
mistakes  encountered  by  AFRL  and  NAWC-AD  during  the  DoD  RCS  Demonstration  Program. 

As  stated  many  times  earlier,  the  main  goal  of  the  range  book  is  to  create  and  organize 
the  RCS  system  documentation  into  a  useful  format  independent  of  the  type  used  by  the  RCS 
range  performing  the  measurements.  While  there  is  a  great  deal  of  latitude  granted  to  the  range 
organizing  the  information,  we  must  be  mindful  of  the  time  constraints  of  the  range  book 
reviewers.  AFRL  and  NAWC-AD’ s  Atlantic  Test  Range  had  the  unfortunate  disadvantage  of 
having  their  range  books  written  before  the  evaluation  criteria  were  formally  released.  It  also 
meant  that  AFRL  and  NAWC-AD  had  many  mid-review  revisions  to  accomplish  because  their 
range  books  were  not  organized  in  exact  line  with  the  evaluation  criteria. 
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The  46  Test  Group  (46TG)  RAMS  VHF/UHF  Measurement  Systems  (RVUMS)  drafted 
their  initial  range  book  much  more  efficiently.  The  book  sections  display  specific  chapters  of 
information  directly  in  parallel  with  the  evaluation  criteria  (i.e.,  criteria  10.1:  Does  the  range 
have  documented  system- wide  calibration  procedures?).  The  RVUMS  range  book  listed  each 
evaluation  criteria,  followed  immediately  by  the  information  needed  to  address  the  criteria.  By 
organizing  in  this  manner,  the  46TG  made  their  range  book  sections  considerably  shorter  in  total 
length,  and  much  more  focused  to  the  points  of  the  evaluation  criteria.  This  subtle  change 
greatly  reduced  the  time  needed  by  the  third  party  reviewers  to  evaluate  the  range  book. 

2.5  The  Range  Book  Reviewers 

The  range  book  reviewers  are  volunteer  experts  from  Government,  academia,  or  industry 
who  have  documented  experience  in  RCS  measurements.  The  entire  certification  process  is 
centered  on  the  concept  of  a  third  party  peer  review.  Consistent  with  ISO  practices,  the 
reviewers  are  specialists  who  understand  the  technical  details  of  RCS  ranges.  In  the  earlier  DoD 
RCS  Demonstration  Program,  no  DoD  employees  were  used  as  reviewers  to  avoid  the 
appearance  of  conflict  of  interest.  Reviewers  for  the  three  DoD  demonstration  ranges  (Figure  2- 
1)  came  from  Boeing  Phantom  Works,  Northrop  Grumman,  EG&G,  Syracuse  Research  Corp., 
Mission  Research  Corp.,  and  the  National  Institute  of  Standards  and  Technology  (NIST). 

The  process  of  becoming  an  RCS  Range  Book  reviewer  is  straightforward.  Those 
interested  in  becoming  reviewers,  must  prepare  two  documents:  (1)  a  one-page  resume  of  RCS 
design  and/or  measurement  experience  demonstrating  a  minimum  of  five  years  of  relevant 
experience  in  RCS  measurements,  and  (2)  a  corporate  letter  of  commitment  from  the  applicant 
reviewer’s  employer  stating  that  the  reviewer  may  participate  in  two  range  reviews  in  three 
years,  and  that  release  time  will  be  granted  the  reviewer  to  support  the  on-site  range  reviews. 

The  resume  and  corporate  commitment  letter  are  transmitted  to  the  RCC/SMSG  Radar 
Committee  Chair.  (As  of  the  date  of  this  publication,  the  Radar  Committee  Chair  is  Mr.  John 
Lewis,  NAWC-AD,  tel.  #  301-342-1193,  e-mail  lewisja@navair.navy.mil.)  The  Radar 
Committee  Chair  reviews  and  approves  the  resume  with  the  SMSG  Radar  Committee.  The 
applicant  reviewer’s  name  is  then  added  to  the  pool  of  RCS  reviewer  candidates  available  for 
range  book  reviews,  as  discussed  previously  in  this  report.  An  applicant  reviewer  is  selected  for 
a  specific  range  book  review  based  on  two  main  criteria:  (1)  the  schedule  availability  of  the 
reviewer  for  a  specific  review  period,  and  (2)  the  lack  of  any  perceived  or  financial  conflict  of 
interest  between  a  reviewer  and  the  range  under  review. 

One  may  ask,  “Why  be  a  reviewer?”  Anyone  with  an  interest  and  technical  background 
in  RCS  measurements  is  strongly  encouraged  to  volunteer  to  be  a  reviewer.  For  ranges 
anticipating  RCS  certification  in  the  future,  it  is  strongly  recommended  that  at  least  one  person 
from  that  organization  volunteer  to  be  a  reviewer  for  another  range.  To  avoid  inefficient,  small- 
dollar  money  transfers  between  companies  in  the  industrial  RCS  certification  program,  we  have 
operated  on  the  quid-pro-quo  foundation  that  if  a  range  wants  to  be  reviewed  (certified),  it  should 
be  prepared  to  review  (certify)  another  range.  In  this  manner,  solid  RCS  experience  and  best 
practices  will  migrate  throughout  the  RCS  measurement  industry,  raising  both  the  overall  quality 
of  measurement  practices  while  instilling  “generally  accepted  measurement  practices” 
throughout  the  RCS  measurement  industry. 
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CHAPTER  3 


FUTURE  RCS  CERTIFICATION  REQUIREMENTS 


3.1  General 

The  primary  emphasis  of  the  DoD  RCS  Demonstration  Program  was  to  improve  the 
quality  of  RCS  data  acquired  at  DoD  RCS  ranges.  The  program  was  designed  to  show  both 
industry  and  the  government  ranges  that  this  was  a  worthwhile,  value-added  quality  assurance 
initiative.  Presently,  there  are  no  DoD  requirements  to  obtain  RCS  certification.  However, 
beginning  on  January  1,  2004,  all  industrial  RCS  measurement  facilities  supplying  deliverable 
RCS  data  to  the  Air  Force  Research  Laboratory  will  be  required  to  obtain  RCS  data  on  a  certified 
RCS  measurement  range.  Thus,  the  voluntary  aspect  of  range  certification  will  quickly  revert  to 
a  requirement  in  2004  for  much  of  the  RCS  measurement  industry.  This  is  more  than  sufficient 
time  for  most  ranges  to  create  an  RCS  Range  Book  in  accordance  with  Z-540.  In  addition,  there 
is  much  documentation  available  from  the  demonstration  program  that  can  help  any  range 
quickly  come  up  to  speed  on  the  certification  process.  Since  industry  is  actively  working  to 
comply,  it  only  makes  sense  for  the  remaining  U.S.  Army,  Navy,  and  Air  Force  facilities 
performing  RCS  measurements  to  comply  with  the  Z-540  certification  requirements  as  well. 

3.2  Certifying  to  Z-540 

Certifying  an  RCS  range  to  the  Z-540  standard  is  an  excellent  approach  to  organizing, 
maintaining,  and  presenting  RCS  range  information.  Any  RCS  Range  Book  that  passes  the  third 
party  review  to  become  “certified”  will  produce  a  document  useful  to  the  range  and  range 
customer  alike.  In  addition,  it  will  permanently  capture  critical  process  information  that  is 
retained  by  the  range  even  as  personnel  turn  over  or  are  assigned  to  other  duties. 

We  believe  that  with  the  overall  emphasis  on  quality  systems  and  ISO  9000  in  the 
commercial  sectors,  Z-540  (ISO-25)  is  an  appropriate  technical  standard  for  RCS  measurement 
facilities.  Certifying  RCS  ranges  to  this  standard  should  greatly  reduce  measurement  errors  and 
generally  improve  the  operational  efficiencies  of  all  RCS  ranges.  In  addition,  certification  will 
allow  a  range  to  assemble,  in  one  paper  document  or  internal  web-site,  all  the  relevant 
information  needed  to  operate  an  RCS  measurement  facility.  This  should  help  a  range  to  sustain 
measurement  quality  during  periods  of  personnel  turnover,  since  the  processes  and  procedures 
will  be  there  even  after  the  employee(s)  has  left.  In  light  of  career  mobility,  and  a  national  aging 
technical  workforce,  this  benefit  alone  should  be  a  powerful  incentive  for  any  organization  to  get 
and  stay  RCS  certified. 

3.3  RCS  Certification 

Probably  the  most  surprising  aspect  of  the  DoD  RCS  Demonstration  Program  is  how 
robust  the  initial  draft  RCS  certification  process  turned  out  to  be.  Despite  early  teething 
problems  and  the  need  to  train  the  RCS  reviewers  on  the  process,  in  the  end  the  evaluators  and 
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ranges  both  agreed  the  process  was  fair,  impartial,  and  value  added.  In  addition,  an  examination 
of  the  review  process  (as  a  whole)  and  the  individual  evaluation  criteria  after  the  Advanced 
Compact  Range  (ACR)  and  Atlantic  Test  Range  (ATR)  reviews  revealed  that  only  minor 
changes  to  the  RCS  certification  process  were  needed.  For  this  reason,  the  RCS  industrial 
certification  program  is  nearly  identical  to  the  DoD  program  recently  completed. 

As  for  side  benefits,  AFRL  and  ATR  range  personnel  who  participated  in  the  DoD  RCS 
Demonstration  Program  remarked  how  much  easier  it  was  to  find  often-used  information  within 
the  structure  of  their  revised  range  books,  as  opposed  to  the  prior  “ad  hoc”  random  access 
methods.  As  mentioned  earlier,  with  the  measurement  community  getting  older,  smaller,  and 
more  mobile,  the  resulting  consolidation  of  documentation  should  help  alleviate  training  and 
sustainment  problems  due  to  personnel  turnover,  thereby  helping  a  range  sustain  its  high-quality 
measurement  efforts. 

With  more  attention  being  paid  within  DoD  to  ISO  9000-like  processes,  the  DoD  RCS 
Demonstration  Program  for  RCS  certification  is  a  relevant  way  for  a  range  to  preemptively  hold 
off  management  dictates  concerning  quality  processes.  The  RCS  review  process  works,  and  the 
results  make  creating  the  range  book  worth  the  effort.  It  is  our  collective  opinion  that  it’s  time 
for  this  RCS  certification  effort  to  become  the  norm.  Our  RCS  customers  will  only  expect 
quality  to  improve  in  the  future,  and  it’s  our  job  to  deliver  the  promise. 

3.4  The  Future  for  Z-540 

Another  conclusion  of  the  DoD  RCS  Demonstration  Program  was  that  its  certification 
approach  is  general.  There  is  nothing  about  Z-540  that  restricts  the  work  to  radio  frequency  RCS 
ranges.  The  processes  and  procedures  are  general  enough  to  cover  all  types  of  scientific 
measurements,  including  laser  cross  section,  infrared  emissions  and  emissivity  testing,  radio 
frequency  property  of  materials  testing,  antenna  testing,  pressure  and  temperature  testing,  and  so 
forth.  If  government  laboratories  and  measurement  facilities  hope  to  continue  to  sustain  the 
finest  test  capability  possible  for  supporting  DoD’s  future  weapons  system  test  and  evaluation 
requirements,  the  Z-540  standard  should  be  looked  at  in  a  broader  context  for  application  to  these 
other  diverse  measurement  problems. 

It  is  clear  to  the  RC C/S  MSG  Radar  Committee  that  this  standard  may  revolutionize  the 
way  DoD  maintains  and  documents  its  in-house  test  capabilities.  We  strongly  recommend  that 
the  RCC  look  for  other  opportunities  and  measurement  system  applications  for  Z-540  to 
establish  and  document  quality  systems  for  all  technical  measurement  systems  used  in  support  of 
the  acquisition  of  future  U.S.  military  hardware. 


3-2 


APPENDIX  I 

RCS  RANGE  BOOK 
FORMAT 


1-1 


APPENDIX  I 


RCS  RANGE  BOOK  FORMAT 


1.0  Introduction 

This  appendix  contains  guidelines  for  the  construction  of  the  RCS  Range  Book  used  to 
document  range  characteristics  and  procedures  in  compliance  with  RCS  range  certification 
requirements.  We  describe  a  common  organization  for  range  documentation  based  on 
ANSI/NCSL  Z-540- 1-1994  Standard  [2]  (referred  to  herein  as  Z-540)  and  discuss  the  material 
the  range  book  should  contain.  Within  each  range  book  section,  however,  company  format  may 
be  used  at  the  discretion  of  the  individual  range.  In  this  manner,  substantial  costs  for 
reformatting  information  is  avoided,  while  at  the  same  time  the  range  books  will  have  the  same 
overall  organization  from  range  to  range. 

RCS  ranges  may  consider  developing  their  range  books  as  electronic  documents  rather 
than  as  paper  documents.  While  a  conventional  paper  copy  certainly  meets  all  requirements  of 
Z-540,  an  electronic  version  available  on  a  network  server  or  CD-ROM  may  be  easier  to  develop 
and  maintain. 

The  RCS  Range  Book  should  serve  as  an  important  general  reference  for  range  personnel 
and  customers  and  will  play  an  important  part  in  the  certification  review.  We  believe  that 
organizing  range  documentation  and  making  it  easy  to  access  will  produce  better 
communications  and  more  consistent  practice  within  each  range.  Hence,  great  care  should  be 
exercised  in  developing  and  maintaining  the  range  book. 

2.0  RCS  Range  Book  Outline 

The  range  book  is  organized  to  parallel  Part  I  of  the  Z-540  standard  with  additional 
sections  added  at  the  end  to  address  important  subsections  of  Z-540’ s  Section  5.  Table  1-1 
outlines  the  range  book  Table  of  Contents. 
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TABLE  1-1 


RCS RANGE  BOOK 
TABLE  OF  CONTENTS 


SECTION  TITLE  PAGE 


_L _  Introduction  and  Endorsement _ 

2  References 

_3 _ Glossary _ 

4  Organization  and  Management 

_5 _ Quality  System,  Audit,  and  Review _ 

6  Personnel 

_7 _ Accommodation  and  Environment _ 

_8 _ Equipment  and  Reference  Materials _ 

_9 _ Measurement  Traceability  and  Calibration 

10  _ Calibration  Methods _ 

11  _ Handling  of  Calibration  Items _ 

12  _ Records _ 

13  Certificates  and  Reports 

14  Subcontracting  of  Calibration 

15  _ Outside  Support  Services  and  Suppliers 

16  Complaints 

17  Interrange  Comparison  Programs 

18  _ Data  Processing  Procedures _ 

19  Range-Specific  Uncertainty  Analysis 

20  Ongoing  Research,  Planned  Improvements 


To  aid  in  document  maintenance,  pages  should  be  numbered  by  section.  In  addition  to 
the  main  table  of  contents,  each  section  may  begin  with  a  brief  section  table  of  contents  listing 
the  titles  of  the  major  subsections,  and  most  importantly,  the  date  of  the  most  recent  revision.  A 
sample  section  table  of  contents  is  shown  in  Table  1-2. 
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TABLE  1-2 

SECTION  TABLE  OF  CONTENTS 

SECTION  10 

CALIBRATION  METHODS 

Last  Revised:  23  July  1997 

CONTENT 

PAGE 

General  Description 

10-1 

Measurement  Procedures 

10-2 

Calibration  Procedures 

10-8 

List  of  Calibration  Objects 

10-14 

Monostatic  Calibration  Tables 

10-15 

Nominal  Calibration  Results  -  Rotators 

10-17 

Nominal  Calibration  Results  -  Foam 

10-19 

Uncertainty  Policy,  Estimation  Procedures 

10-21 

3.0  Range  Book  Content 

The  following  provides  guidelines  for  the  construction  of  the  range  book  in  accordance 
with  ANSI/NCSL  Z-540-1-1994.  Before  submitting  a  range  book  for  review,  range  book 
authors  should  use  the  evaluation  criteria  in  Appendix  II  to  perform  a  “self  review”  of  each 
section.  The  evaluation  criteria  will  be  used  by  the  RCS  Certification  Review  Committee  to 
grade  the  range  book. 


/While  there  is  a  great  deal  of  latitude  granted  to  the  range  book  authors,  we 
must  be  mindful  of  the  time  constraints  of  the  range  book  reviewers.  To 
w  expedite  the  grading  process  for  the  reviewers,  range  book  authors  may  list 
each  evaluation  criteria 
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1  -  Introduction  and  Endorsement  (see  Appendix  II,  Section  1,  Evaluation  Criteria):  This  section 
should  contain  a  brief  statement  of  compliance  with  the  standards  set  by  this  document  in 
accordance  with  ANSI/NCSL  Z-540- 1-1994,  as  well  as  a  policy  statement  regarding  the 
organization's  commitment  to  continuous  quality  improvement.  The  range  book  should  be 
signed  as  “approved”  by  an  appropriate  manager  or  director  at  or  near  the  top  of  the  chain  of 
command  of  the  range  quality  manager  or  technical  lead  engineer.  The  exact  format  and 
wording  may  be  customized  by  the  organization. 

2  -  References  (see  Appendix  II,  Section  2,  Evaluation  Criteria):  This  section  should  list  the 
documents  cited  in  the  range  book.  In  particular,  it  may  be  convenient  to  include  stand-alone 
reports  covering  various  aspects  of  range  operations  (operational  security,  measurement 
procedures,  uncertainty  analysis  procedures,  equipment  documentation,  etc.),  which  could  be 
included  in  the  range  book  by  reference  only. 

3  -  Glossary  (see  Appendix  II,  Section  3,  Evaluation  Criteria):  This  section  should  list  and 
define  the  specialized  terms  and  acronyms  used  in  the  range  book.  All  acronyms  should  be 
defined  upon  first  use.  Specialized  terms  related  to  the  acquisition  and  processing  of  RCS  data 
should  also  be  defined.  The  information  in  this  section  should  aid  the  reviewers  in  translating 
technical  jargon  from  the  body  of  the  report. 

4  -  Organization  and  Management  (see  Appendix  II,  Section  4,  Evaluation  Criteria):  This 
section  should  fully  define  the  organization  that  is  chartered  to  operate  the  range.  The 
management  hierarchy  from  the  range  line  technicians  and  engineers  up  through  the  highest 
levels  of  range  management  should  be  defined.  Lead  personnel  and  approved  signatories  should 
be  identified.  The  range  quality  manager  and  technical  managers  must  be  identified.  Ideally, 
work  phone  numbers  should  be  available  for  all  critical  range  personnel.  Standard  company 
organization  charts  are  acceptable.  Also,  a  policy  statement  on  customer  confidentiality  and  or 
proprietary  rights  protection  must  be  included  here. 

5  -  Quality  System,  Audit,  and  Review  (see  Appendix  II,  Section  5,  Evaluation  Criteria):  The 
purpose  of  this  section  is  to  ensure  that  a  complete  overview  of  the  company  quality  system  and 
self-auditing  process  is  provided.  This  section  should  contain  procedures  for  the  control  and 
maintenance  of  documentation,  arrangements  for  ensuring  that  the  range  reviews  all  new  work  to 
ensure  that  it  has  the  appropriate  facilities  and  resources  before  commencing  such  work, 
management  arrangements  for  permitting  departures  from  documented  policies  and  procedures 
or  from  standard  specifications,  procedures  for  protecting  confidentiality  and  proprietary  rights, 
procedures  for  audit  and  review,  procedures  for  a  management  quality  audit  at  least  once  a  year, 
procedures  for  documentation  of  all  audit  processes,  and  procedures  for  tracking  and  monitoring 
the  quality  of  the  primary  RCS  calibration  data.  The  RCS  Certification  Review  Committee’s 
Certification  Report  should  be  attached  here  upon  completion  of  the  certification  review  process. 

6  -  Personnel  (  see  Appendix  II,  Section  6,  Evaluation  Criteria):  This  section  should  contain  the 
qualifications  of  all  personnel  assigned  to  the  range.  Specialized  training,  training  certificates,  or 
other  range-related  qualifications  should  appear  in  this  section.  Company  or  resume  format  may 
be  used.  If  an  organization,  due  to  personnel  policy  or  union  concerns,  treats  their  records  as 
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“company  sensitive”  or  “company  confidential,”  the  range  book  should  so  state,  and  indicate 
what  individual  and  organization  has  these  records,  and  who  has  the  right  to  review  and  update 
them.  It  is  the  responsibility  of  the  range  to  certify  that  their  staff  has  the  qualifications  to  do 
their  jobs.  Updated  training  records  are  one  indicator  of  the  currency  of  training.  If  the 
reviewers  are  not  allowed  to  see  these  records,  the  range  technical  manager  may  certify,  in  a 
signed  statement  herein,  that  such  records  exist  in  a  remote  location,  and  that  these  records 
demonstrate  that  the  employees  meet  the  minimum  qualification  standards  for  their  respective 
jobs. 

7  -  Accommodation  and  Environment  (see  Appendix  II,  Section  7,  Evaluation  Criteria):  This 
section  should  document  the  range  environment  as  well  as  any  and  all  parameters  of  the  range 
environment  that  may  affect  the  outcome  of  measurements,  and  thereby  should  be  monitored. 
Some  environmental  variables  that  could  affect  radar  cross  section  measurements  include: 
electromagnetic  interference,  temperature  variations,  mechanical  vibrations,  and  seasonal 
variations  in  vegetation,  sea  state,  moisture  levels,  winds,  precipitation,  inversions,  etc. 
(Depending  on  the  complexity  of  the  range,  this  section  may  or  may  not  be  significant.) 

8  -  Equipment  and  Reference  Materials  (see  Appendix  II,  Section  8,  Evaluation  Criteria): 
Configuration  control  is  essential  for  maintaining  a  repeatable,  quality  RCS  system.  Therefore, 
this  section  of  the  range  book  may  be  one  of  the  most  significant.  Here  all  electromechanical 
equipment  that  makes  up  the  RCS  range  should  be  identified.  References  to  appropriate 
equipment  manuals  should  be  given  in  sufficient  detail  so  that  a  qualified  technician  can  trace  the 
major  subsystems  of  the  range.  Maintenance  and  calibration  histories  should  be  kept  for  critical 
components,  and  complete  system  configuration  should  be  tracked.  The  procedures  for 
identifying  equipment,  maintenance,  and  configuration  control  should  be  documented  here. 
Personnel  responsible  for  following  these  procedures  should  be  clearly  identified.  For  electronic 
(digital/internet)  range  books,  it  is  not  essential  to  include  an  electronic  version  of  critical  range 
equipment.  However,  the  titles  and  exact  locations  of  the  referenced  documents  must  be 
included. 

9  -  Measurement  Traceability  and  Calibration  (see  Appendix  II,  Section  9,  Evaluation  Criteria): 
This  section  should  complement  Section  8  as  it  specifically  relates  to  primary  RCS  calibration. 
The  range  will  document  its  calibration  program  that  ensures  that  all  equipment  critical  to  RCS 
measurement  calibrations  operates  within  certified  performance  limits.  Equipment  and  system 
calibration  intervals  should  be  clearly  established.  Whenever  possible,  traceability  of  system 
components  and  of  total  system  calibration  to  a  national  standard  should  be  maintained. 
Calibration  certificates  and  reports  should  be  referenced.  System  configuration  used  in  RCS 
calibrations  should  be  tracked  using  the  method  documented  in  Section  8.  The  system 
configuration  used  in  interrange  comparisons  should  be  clearly  documented  together  with  the 
results  of  calibration  comparisons.  System  components  tracked  under  Section  8  need  only  be 
referenced  here. 

10  -  Calibration  Methods  (see  Appendix  II,  Section  10,  Evaluation  Criteria):  Both  measurement 
and  calibration  procedures  should  be  fully  documented.  Separate  documentation  should  be 
available  for  wideband  calibration  and  measurement  procedures,  monostatic  and  bistatic 
(multistatic)  procedures,  and  polarimetric  (reciprocal  or  nonreciprocal)  calibration  procedures. 
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Procedures  for  deriving  or  assigning  an  RCS  value  to  a  primary  target  through  the  use  of 
“transfer”  calibration  standards  should  be  documented  in  this  section.  Calibration  and 
measurement  results  to  support  the  validity  of  procedures  used  should  be  displayed  or 
referenced.  The  validity  of  such  results  should  be  supported  by  stated  uncertainty  bounds 
obtained  through  well-defined  uncertainty  procedures  appropriately  referenced.  Calibration 
intervals  should  be  clearly  established. 

11  -  Handling  of  Calibration  Items  (see  Appendix  II,  Section  11,  Evaluation  Criteria):  This 
section  should  summarize  how  primary  calibration  items  are  protected  during  storage,  handling, 
and  use.  Mechanical  tolerance  certificates  or  techniques  for  verifying  tolerances  of  primary 
standards  are  appropriately  placed  in  this  section. 

12  -  Records  (see  Appendix  II,  Section  12,  Evaluation  Criteria):  This  section  describes  the 
range's  procedures  for  creating  and  maintaining  records  of  all  aspects  of  range  operations. 
Procedures  for  maintaining  calibration  records,  measurement  records,  system  configuration,  etc. 
are  especially  important.  Procedures  to  record  interrange  comparison  results  should  be  clearly 
defined. 

13  -  Certificates  and  Reports  (see  Appendix  II,  Section  13,  Evaluation  Criteria):  This  section 
should  define  the  standard  reporting  formats  used  by  the  ranges  when  reporting  RCS 
measurements  to  customers.  While  any  report  can  be  specialized  to  the  requirements  of  the 
customer,  this  section  should  provide  the  baseline  information  that  will  always  be  available  in  a 
test  report.  A  range  may  use  its  own  format,  provided  that  the  minimum  information  required  by 
the  Evaluation  Criteria  in  Appendix  II,  Section  13,  is  included. 

14  -  Subcontracting  of  Calibration  (see  Appendix  II,  Section  14,  Evaluation  Criteria):  This 
section  details  aspects  of  the  RCS  calibration  process  that  are  subcontracted.  All  subcontracted 
work  must  satisfy  the  standards  set  by  this  document  in  accordance  with  ANSENCSL  Z-540-1- 
1994.  The  main  intent  of  this  section  is  to  address  calibration  activities  sent  to  locations  or 
personnel  not  located  on-site  within  the  range  facility.  When  on-site  contractors  share  complete 
responsibility  for  operating  a  range  facility,  as  is  often  the  case,  we  refer  to  this  situation  as  an 
integrated  product  team.  For  example,  AFRL  hires  an  on-site  contractor  to  assist  in  range 
operations.  This  is  not  considered  subcontracting.  However,  if  a  primary  range  relies  on  an  off¬ 
site  secondary  RCS  range  for  calibration  data  or  other  measurements  applicable  to  the  primary 
range,  this  situation  would  be  treated  as  a  subcontracted  measurement,  and  Section  14  would 
apply.  For  most  ranges,  these  criteria  probably  will  not  apply. 

15  -  Outside  Support  Services  and  Suppliers  (see  Appendix  II,  Section  15,  Evaluation  Criteria): 
This  section  should  document  any  services  or  suppliers  used  to  produce  calibrated  RCS  data. 

This  section  complements  Section  14.  For  instance,  if  a  range  purchases  an  8-inch  calibration 
sphere  from  a  vendor  for  primary  calibration,  this  section  would  document  the  needed  tolerances 
of  such  a  standard,  as  well  as  the  delivered  tolerance  of  the  calibration  item.  As  an  additional 
example,  if  radar  absorbing  material  (RAM)  is  used  on  a  calibrated  target  measurement  and 
absolute  RCS  data  is  reported,  it  would  be  appropriate  to  create  records  of  where  the  RAM  was 
purchased,  what  characteristics  were  expected  and  delivered,  and  what,  if  any,  repeatability 
would  result  from  reusing  the  same  material  samples  on  the  same  target  location. 
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16  -  Complaints  (see  Appendix  II,  Section  16,  Evaluation  Criteria):  This  section  should 
document  the  formal  complaint  procedure  used  to  resolve  disputes  between  the  range  and  the 
customer.  The  complaint  record  keeping  system  also  should  be  defined  or  referenced  here. 

17  -  Interrange  Comparison  Programs  (see  Appendix  II,  Section  17,  Evaluation  Criteria):  This 
section  shall  include  the  company  policy  regarding  interrange  comparisons.  Records  of 
participation  in  calibration  interrange  comparisons,  the  results  obtained,  and  conclusions  drawn 
from  such  studies  are  to  be  presented  or  referenced  here.  If  the  information  is  already  included 
in  Section  10,  a  reference  to  Section  10  is  adequate.  It  is  expected  that  interrange  comparisons 
will  be  used  regularly  to  improve  measurement  and  calibration  quality  at  the  range.  An 
interrange  comparison  schedule  should  be  provided. 


/NOTE 

/ 


W 


Participation  in  a  national  interrange  comparison  study  is  required  for 
certification. 


Publication  of  interrange  comparison  results  is  strongly  encouraged,  and  such  publications 
should  be  referenced  here.  Procedures  to  record  interrange  comparison  results  should  be  clearly 
defined. 

18  -  Data  Processing  Procedures  (see  Appendix  n,  Section  18,  Evaluation  Criteria):  Important 
details  of  processing  measurement  and  calibration  data  should  be  documented  here.  Sample 
results  of  processing  should  be  included.  Standard  data  processing  procedures  should  be 
referenced,  and  the  validity  of  innovative  processing  should  be  documented.  To  evaluate  the 
validity  of  its  data  processing  procedures,  the  range  should  use  a  standard  data  set  acceptable  to 
the  RCS  industry.  This  standard  data  set  should  be  used  by  the  range  to  perform  interrange 
comparison  of  its  processing  algorithms  with  those  used  by  other  ranges.  This  section  should 
strongly  support  and  complement  the  material  in  Section  17  above. 

19  -  Range-Specific  Uncertainty  Analysis  (see  Appendix  II,  Section  19,  Evaluation  Criteria): 
Policies  and  procedures  for  establishing  range  uncertainties  need  to  be  completely  documented 
or  referenced  here.  A  sample  uncertainty  table  together  with  system  parameters  or  other 
adequate  uncertainty  method  should  be  linked  or  available  in  this  section.  One  approach  to 
satisfy  this  requirement  is  the  “Report  of  Measurement”  described  in  [15].  Any  system 
parameters  should  be  displayed  here.  Any  scientifically-based  uncertainty  analysis  is  acceptable, 
providing  all  appropriate  assumptions  and/or  exclusions  regarding  the  parameters  comprising  the 
analysis  are  clearly  documented. 

20  -  Ongoing  Research,  Planned  Improvements  (see  Appendix  II,  Section  20,  Evaluation 
Criteria):  The  purpose  of  this  section  is  to  briefly  summarize  the  three-year  look  ahead  regarding 
any  range  research  and/or  other  activities  designed  to  improve  range  data  quality,  efficiency, 
repeatability,  or  traceability.  A  list  of  desirable  research  areas  to  be  conducted  in  the  future  to 
improve  specific  and  known  deficiencies  on  the  range  or  in  the  RCS  industry  should  also  be 
provided,  as  well  as  a  single  top-level  roadmap.  Ongoing  plans  for  future  upgrades  of  equipment 
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should  be  included.  Such  research  information  will  allow  customers  to  quickly  identify  and 
evaluate  ongoing  range  improvements  in  the  context  of  their  current  or  planned  use  of  the  range. 
It  also  may  offer  the  customer  an  opportunity  to  cost  share  or  jointly  sponsor  range  research  of 
interest  to  the  customer. 

4.0  Summary 

The  RCS  Range  Book  will  be  reviewed  by  the  RCS  Certification  Review  Committee  and 
graded  according  to  the  evaluation  criteria  in  Appendix  II.  Once  completed,  it  is  the 
responsibility  of  the  range  to  keep  the  range  book  current.  Fortunately,  as  mentioned  previously, 
this  will  involve  revising  individual  sections  only,  as  needed.  Since  the  book  is  numbered  by 
section,  it  is  modular  in  design,  making  updates  a  fairly  straightforward  process. 
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APPENDIX  II 


EVALUATION  CRITERIA 
FOR 

RCS  RANGE  BOOK 


1.0  Background 

This  appendix  summarizes  a  joint  in-house  research  and  development  activity  performed 
by  the  Air  Force,  Navy,  Range  Commanders  Council,  and  the  National  Institute  of  Standards  and 
Technology  (NIST)1.  From  1997-1999,  the  Range  Commanders  Council  Signature 
Measurement  Standards  Group  (RCC/SMSG)  and  NIST  worked  on  a  joint  Department  of 
Defense  (DoD)  RCS  Demonstration  Program  of  a  new  radar  cross  section  measurement  range 
quality  documentation  standard  based  on  the  American  National  Standards  Institute/National 
Calibration  Standards  Laboratory  ANSI/NCSL  Z-540- 1-1994  Standard  [2]  (referred  to  herein  as 
Z-540)  as  customized  to  the  measurement  of  radar  cross  section. 

2.0  Executive  Summary 

The  purpose  of  the  DoD  RCS  Demonstration  Program  was  to  improve  the  long-term 
quality  of  radar  cross  section  measurement  data  taken  in  the  United  States  by  implementing  a 
rigorous  and  uniform  range  system  documentation  standard  for  all  facilities  supplying  RCS  data 
to  the  Department  of  Defense.  At  the  initiation  of  this  demonstration  program,  it  was  decided 
that  the  commercial  standard  ANSI/NCSL  Z-540- 1-1994  was  an  appropriate  structure  to  use  in 
organizing  and  presenting  this  range  documentation  standard.  The  DoD  facilities  participating  in 
this  demonstration  program  were  the  Air  Force  Research  Laboratory,  USAF  46th  Test  Group, 
and  the  Navy’s  Dynamic  Air  RCS  Measurement  Facility2.  These  facilities  agreed  to  organize 
their  range  documentation  around  a  quality  standard  written  by  the  National  Institute  of 
Standards  and  Technology  (NIST)  and  the  Air  Force  Research  Laboratory  titled  Quality 
Assurance  Handbook  For  Radar  Cross  Section  For  simplicity,  the  range  documentation  is  now 
referred  to  as  the  RCS  Range  Book,  the  definitive  document  on  the  operations  of  the  RCS  range 
facility. 


As  part  of  the  DoD  RCS  Demonstration  Program,  the  three  participating  ranges  agreed  to 
have  their  range  books  reviewed  by  an  outside  third  party  review  panel.  This  review  panel 
consisted  of  three  experienced  and  qualified  individuals  from  the  RCS  measurement  community. 
The  reviewer’s  job  was  to  assess  how  well  the  range  books  complied  with  the  draft  national 
quality  assurance  standard. 


1  Signature  Technology  Office,  Air  Force  Research  Laboratory  (AFRL),  Wright  Patterson  Air  Force  Base;  Naval  Air  Warfare 
Center  Aircraft  Division,  Patuxent  River,  Maryland;  Signature  Measurement  Standards  Group  of  the  Range  Commanders 

Council  (RCC/SMSG);  and  the  National  Institute  of  Standards  and  Technology  (NIST),  Boulder,  Colorado. 

2 

"  Advanced  Compact  Range,  Wright  Patterson  Air  Force  Base  (WPAFB),  OH;  RAMS  Facility,  Holloman  Air  Force  Base 
(HAFB),  NM;  and  Atlantic  Test  Range,  Naval  Air  Warfare  Center,  Patuxent  River,  MD 
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The  purpose  of  this  appendix  is  to  assist  the  range  book  reviewers  and  the  ranges  by 
providing  a  section-by-section  set  of  evaluation  criteria  for  use  in  the  range  review  process.  The 
original  quality  assurance  standard  identified  20  distinct  chapters  of  information  needed  to 
comply  with  the  intent  of  the  Z-540  standard.  This  shorter  version  matches  specific  evaluation 
criteria  with  each  of  these  sections  using  a  loose  checklist-like  format. 

As  with  the  range  book,  these  evaluation  criteria  were  drafted  to  help  evaluate  the  quality 
of  the  range  books  in  accordance  with  the  Z-540  standard.  If  the  reader  finds  areas  where  these 
criteria  can  be  improved  or  simplified,  we  welcome  your  comments.  Please  send  any  comments 
to  the  Air  Force  Research  Laboratory  Signature  Technology  Office,  Attn:  Dr.  Brian  Kent, 
brian.kent@wpafb.af.mil  or  fax  937-656-7074. 
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RCS  RANGE  BOOK 
CERTIFICATION  REPORT 

GRADING  AND  PASSING  CRITERIA 


GRADING  CRITERIA 


The  following  scale  applies  to  ALL  grading  criteria: 


S  =  Fully  Satisfactory  -  meets  or  exceeds  standard  (no  reviewer  comments  required) 

N  =  Needs  Improvement  -  marginally  satisfactory  but  needs  improvement 

(marginally  meets  standard  but  may  be  improved  using  mandatory  reviewer  comments  as 
guidance) 

U  =  Unsatisfactory  -  (does  not  meet  standard  and  should  be  redone  using  mandatory 
reviewer  comments  as  guidance) 

N/A  =  Not  Applicable  -  (reviewer  comments  needed  if  the  reason  for  the  N/A  grade  is  not 
obvious) 


All  range  books  will  be  graded  by  three  reviewers.  Composite  grades,  defined  as  the  combined 
grades  from  the  three  reviewers,  will  be  derived  as  follows: 


Individual  Grades  from 
the  Three  Reviewers 

Composite  Grade 

Reviewer  Comments 

S,S,N  -or-  S,S,U 

S 

N  or  U  minority  comments 
required 

S,N,N  -or-  S,N,U  -or-  N,N,U 

N 

N  or  U  comments  required 

S,U,U  -or-  N,U,U  -or-  U,U,U 

U 

majority  comments  required 
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PASSING  CRITERIA 


Each  range  book  section  carries  a  priority  based  on  the  impact  it  has  on  the  quality,  accuracy, 
and  repeatability  of  RCS  range  measurements.  The  range  book  sections  have  been  divided  into 
the  following  priorities: 

Priority  1  -  Sections  9,  10,  11,  17,  18,  and  19 

Priority  2  -  Sections  5,  8,  13,  and  20 

Priority  3  -  Sections  1,  2,  3,  4,  6,  7,  12,  14,  15,  and  16 

A  PASSING  Certification  Report  shall  meet  all  the  following  criteria: 

The  Priority  1  sections  altogether  shall  receive  a  minimum  composite  grade  of  85%  Satisfactory 
(S).  The  balance  shall  be  graded  Needs  Improvement  (N);  no  Unsatisfactory  (U)  grades  are 
allowed  (e.g.,  87%S  and  13%N  passes  the  Priority  1  criteria). 

The  Priority  2  sections  altogether  shall  receive  a  minimum  composite  grade  of  75%  Satisfactory 
(S)  and  not  more  than  10%  Unsatisfactory  (U)  with  the  balance  being  Needs  Improvement  (N) 
(e.g.,  77%S,  8%U,  and  15%N  passes  the  Priority  2  criteria). 

The  Priority  3  sections  altogether  shall  receive  a  minimum  composite  grade  of  65%  Satisfactory 
(S)  and  not  more  than  20%  Unsatisfactory  (U)  with  the  balance  being  Needs  Improvement  (N) 
(e.g.,  69%S,  18%U,  and  13%N  would  pass  the  Priority  3  criteria). 


y  NOTE 


"N/A"  grades  are  excluded  from  the  percentage  count.  Standard 
round-up  techniques  will  be  used  (i.e.,  84.5%=85%,  84.49%=84%). 


A  final  grade  will  be  assigned  to  the  overall  Certification  Report  which  will  be  either: 

PASSING 

or 

REVISION  REQUIRED  BEFORE  PASSING 

A  Certification  Report  receiving  a  PASSING  grade  will  have  met  all  the  grading  criteria  above. 
A  report  receiving  the  REVISION  REQUIRED  BEFORE  PASSING  grade  may  be  upgraded 
to  PASSING  upon  closure  of  the  requisite  number  of  graded  items  that  are  under  the  passing 
criteria.  Note  that  at  the  conclusion  of  the  on-site  review,  the  review  committee  will  share  with 
the  range  where  it  stands  at  that  point  and  will  give  the  range  ample  opportunity  to  respond  to 
issues.  If  closure  of  these  issues  takes  longer  than  30  days,  the  range  may  ask  for  an  additional 
30-day  extension  to  correct  the  requisite  number  of  outstanding  issues. 
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RCS  RANGE  BOOK 
EVALUATION  CRITERIA 


SECTION  1  -  INTRODUCTION  AND  ENDORSEMENT  (PRIORITY  3) 

This  section  should  contain  a  brief  statement  of  compliance  with  the  standards  set  by  this 
document  in  accordance  with  ANSI/NCSL  Z-540-1-1994,  as  well  as  a  clear  policy  statement 
regarding  the  organization's  commitment  to  continuous  quality  improvement.  The  range  book 
should  be  signed  as  “approved”  by  an  appropriate  manager  or  director  at  or  near  the  top  of  the 
reporting  chain  of  the  range  quality  manager  or  technical  lead  engineer.  The  exact  format  and 
wording  may  be  customized  to  suit  the  organization. 

Evaluation  Criteria  (See  Page  2): 

_ 1.1:  Does  the  range  book  identify  the  host  organization? 

_ 1.2:  Is  the  endorsement  signed  by  an  appropriate  manager  or  director  who  has  the 

responsibility  and  authority  to  direct  changes  and  corrections  should  range  or  product 
deficiencies  be  found?  (Alternate  location  for  section  5.2f.) 

_ 1.3:  Is  there  a  clear  commitment  to  quality  in  the  introduction  and  endorsement? 

_ On-site  Review:  Ask  to  meet  the  endorser.  Is  the  endorser  clearly  committed  to  the 

quality  process? 


SECTION  2  -  REFERENCES  (PRIORITY  3) 

This  section  should  list  the  documents  cited  in  the  range  book.  In  particular,  it  may  be 
convenient  to  include  stand-alone  reports  covering  various  aspects  of  range  operations 
(operational  security,  measurement  procedures,  uncertainty  analysis  procedures,  equipment 
documentation,  etc.),  which  could  be  included  in  the  range  book  by  reference  only. 

Evaluation  Criteria: 

_ 2.1:  Does  the  range  book  include  a  list  of  documents  and  references  external  to  the  range  book? 

_ _  2.2:  Are  all  references  unambiguous  enough  to  identify  the  documents? 

_ .  2.3:  Are  the  documents  listed  cross-referenced  or  linked  to  the  appropriate  range  book  section? 

_ On-site  Review:  Ask  to  see  five  documents  at  random  on  the  list. 


SECTION  3  -  GLOSSARY  (PRIORITY  3) 

This  section  should  list  and  define  the  specialized  terms  and  acronyms  used  in  the  range  book. 
All  acronyms  should  be  defined  upon  first  use.  Specialized  terms  related  to  the  acquisition  and 
processing  of  RCS  data  should  also  be  defined.  The  information  in  this  section  should  aid  the 
reviewers  in  translating  technical  jargon  from  the  body  of  the  report. 
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Evaluation  Criteria: 


3.1:  Does  the  range  book  contain  an  appropriate  glossary? 

3.2:  Do  acronyms  (jargon)  occur  in  any  range  book  section  that  are  not  defined 
(immediately)  in  that  section  or  listed  here  in  the  glossary? 

(No  On-site  Review  Criteria) 


)  NOTE 

/ 


Reviewers:  Please  list  any  undefined  acronyms  below. 


/NOTE 


Range  Book  Authors:  After  review,  please  include  in  the  glossary  any 
acronyms  or  terms  identified  by  reviewers  as  undefined. 


SECTION  4  -  ORGANIZATION  AND  MANAGEMENT  STRUCTURE  (PRIORITY  3) 

This  section  should  fully  define  the  organization  that  is  chartered  to  operate  the  range.  The 
management  hierarchy  from  the  range  line  technicians  and  engineers  up  through  the  highest 
levels  of  range  management  should  be  defined.  Lead  personnel  and  approved  signatories  should 
be  identified.  The  range  quality  manager  and  technical  managers  must  be  identified.  Ideally, 
work  phone  numbers  should  be  available  for  all  critical  range  personnel.  Standard  company 
organization  charts  are  acceptable.  Also,  a  policy  statement  on  customer  confidentiality  and  or 
proprietary  rights  protection  must  be  included  here. 

Evaluation  Criteria: 

_ 4.1:  Does  the  range  book  contain  a  complete  reporting  chain  or  organizational  chart  of 

the  range? 

_ 4.2:  Does  the  range  identify  a  range  technical  manager? 

_ 4.3:  Does  the  range  identify  the  range  quality  manager? 

_ 4.4:  Does  the  range  identify  the  management  chain  from  the  technician  level  to  the  highest 

level  of  the  organization? 

_ 4.5:  Does  the  range  quality  manager  have  direct  and  unfettered  access  to  the  individual 

signing  the  endorsement  in  section  1,  Introduction  and  Endorsement? 

_ 4.6:  Is  there  a  clear  policy  on  non-disclosure  and/or  company  proprietary  data  or  information? 

_ 4.7:  If  range  operators,  engineers,  and/or  technicians  are  from  a  different  organization  or 

company  than  the  range  owners,  are  the  respective  roles  and  responsibilities  of  the  two 
organizations  and  their  interfaces  clearly  spelled  out? 
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4.8:  Are  personnel  qualifications  available  or  referenced  in  other  range  book  sections? 
4.9:  In  cases  of  absence,  are  alternate  technical  and  quality  managers  identified? 

On-site  Review:  Ask  range  technicians  to  identify  the  technical  and  quality  managers. 
On-site  Review:  Ask  the  quality  manager  if  he/she  has  direct  access  to  the  endorser  of 
section  1. 

On-site  Review:  Check  to  see  whether  the  range  technicians  and  engineers  know  who 
the  primary  and  alternate  technical  and  quality  managers  are. 

On-site  Review:  Is  the  range  supervision  done  by  person(s)  familiar  with  the  calibration 
methods  and  procedures,  the  objective  of  the  calibration,  and  how  to  assess  the  results? 


SECTION  5  -  QUALITY  SYSTEM,  AUDIT,  AND  REVIEW  (PRIORITY  2) 

The  purpose  of  this  section  is  to  ensure  that  a  complete  overview  of  the  company  quality  system 
and  self-auditing  process  is  provided.  This  section  should  contain  procedures  for  the  control  and 
maintenance  of  documentation,  arrangements  for  ensuring  that  the  range  reviews  all  new  work  to 
ensure  that  it  has  the  appropriate  facilities  and  resources  before  commencing  such  work, 
management  arrangements  for  permitting  departures  from  documented  policies  and  procedures 
or  from  standard  specifications,  procedures  for  protecting  confidentiality  and  proprietary  rights, 
procedures  for  audit  and  review,  procedures  for  a  management  quality  audit  at  least  once  a  year, 
procedures  for  documentation  of  all  audit  processes,  and  procedures  for  tracking  and  monitoring 
the  quality  of  the  primary  RCS  calibration  data.  The  RCS  Certification  Review  Committee 
Certification  Report  should  be  attached  here  upon  completion  of  the  certification  review  process. 


NOTE 


/Note  1:  All  items  of  section  5  need  to  be  addressed  at  some  point  in  the 
range  book.  Because  of  their  importance,  some  of  these  items  are  treated  in 
-  -  .  *  separate  sections  of  the  range  book.  Any  such  information  requested  in 
section  5,  but  appearing  elsewhere  in  the  range  book  must  be  appropriately  documented. 
For  example  section  5.2(a)  appears  in  this  book  in  section  1.  Here  we  include  all  items  that 
are  not  treated  separately,  as  listed  after  the  section  heading.  The  purpose  of  this  section  is 
to  ensure  that  a  complete  overview  of  the  company’s  quality  system  and  its  self-auditing 
process  are  given.  Important  miscellaneous  items  are  to  be  included  here.  We  recommend 
closely  following  the  appropriate  parts  of  section  5  of  the  main  text.  Section  5.2(s)  should 
be  the  section  where  the  certification  audit  report  is  attached. 
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)  NOTE 

/ 


W 


Note  2: 

Z540  5.2e  is  found  in  Range  Book  Section  6 

Z540  Section  5.2f  is  found  in  Range  Book  Section  1 

APPENDIX  I  -  Z540  Section  5.2g  is  found  in  Range  Book  Section  9 

Z540  Sections  5.2h,  j,  1,  m  and  n  are  found  in  Range  Book  Section  10 

Z540  Section  5.2k  is  found  in  Range  Book  Section  11 

Z540  Section  5.2q  is  found  in  Range  Book  Section  16 

Z540  Section  5.2r  is  found  in  Range  Book  Section  4 

Z540  Section  5.2u  is  found  in  Range  Book  Section  19 


NOTE  ^  Note  3:  The  numbering/lettering  sequence  below  follows  Z540. 


Evaluation  Criteria: 


_ 5.2a: 

_ 5.2b: 

_ 5.2c: 

_ 5. 2d: 

_ 5.2i: 

_ 5.2o: 

_ 5.2p: 

_ 5.2s: 


Does  the  range  book  contain  a  top  management  quality  statement  here  or  in  section  1? 
Are  appropriate  organizational  charts  in  place  here  or  in  section  4? 

Are  the  roles  and  responsibilities  of  management,  technical  operations,  support  services, 
and  quality  systems  identified  here  or  in  section  4? 

Does  the  range  book  describe  how  documents  are  controlled  and  referenced? 

Does  the  range  organization  have  a  process  to  assess  whether  it  has  the  technical 
capability  and  resources  (schedule  time)  to  accept  new  work? 

Is  there  a  clearly  established  policy  and  process  when  measurement  discrepancies  are 
seen  or  departures  from  documented  policies  and  procedures  occur? 

Is  there  a  management  policy  for  permitted  departures  from  documented  policies? 

Are  periodic  review  processes  documented? 


/NOTE  /  The  RCS  Certification  Review  Committee  Certification  Report  should 
w  be  attached  here  upon  completion  of  the  review  process. 


5.2-t:  Is  the  range’s  policy  for  establishing  and  changing  calibration  intervals  documented 
for  systems  or  equipment  it  controls? 

5.3:  Are  the  range’s  internal  (annual)  and  external  (every  three  year)  audit  processes 

documented?  Have  previous  audit  deficiencies  been  corrected  by  the  next  annual  audit? 
5.4:  Was  a  management  quality  audit  performed  within  the  previous  12  months? 

5.5:  Are  previous  audits  and  their  findings  a  permanent  part  of  the  range  book? 

5.6:  Does  the  range  have  a  process  for  tracking  and  monitoring  the  quality  of  the  primary 
RCS  calibration  data  for  calibration  artifacts  used  in  the  normal  measurement  process? 
Is  there  a  day-to-day  pass/fail  check  criteria  established  for  calibration  measurements? 
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Have  the  results  been  verified,  either  through  repeated  measurements,  interrange 
comparisons  or  both? 

On-site  Review:  Ask  range  technicians  what  the  pass/fail  criteria  for  calibration  is. 

On-site  Review:  Ask  range  technicians  what  is  done  if  calibrations  depart  from  expectations. 
On-site  Review:  Review  internal  range  process  of  assessing  new  work  (or  a  new  customer 
requirement).  Is  it  complete?  Can  it  easily  be  provided  to  a  customer  on  request? 


SECTION  6  -  PERSONNEL  (PRIORITY  3) 

This  section  should  contain  the  qualifications  of  all  personnel  assigned  to  the  range.  Specialized 
training,  training  certificates,  or  other  range-related  qualifications  should  appear  in  this  section. 
Company  or  resume  format  may  be  used.  If  an  organization,  due  to  personnel  policy  or  union 
concerns,  treats  their  records  as  “company  sensitive”  or  “company  confidential,”  the  range  book 
should  so  state,  and  indicate  what  individual  and  organization  has  these  records,  and  who  has  the 
right  to  review  and  update  them.  It  is  the  responsibility  of  the  range  to  certify  that  their  staff  has  the 
qualifications  to  do  their  jobs.  Updated  training  records  are  one  indicator  of  the  currency  of 
training.  If  the  reviewers  are  not  allowed  to  see  these  records,  the  range  technical  manager  may 
certify,  in  a  signed  statement  herein,  that  such  records  exist  in  a  remote  location,  and  that  these 
records  demonstrate  that  the  employees  meet  the  minimum  qualification  standards  for  their 
respective  jobs.  Note  that  any  training  program  should  not  only  address  day-to-day  competencies, 
but  also  allow  for  technical  growth  to  build  expertise  to  support  future  tasks,  like  those  identified  in 
section  20. 

Evaluation  Criteria: 

_ 6.1:  Does  the  range  employ  an  appropriate  mix  of  technician,  engineering,  and  management 

personnel  with  appropriate  experience  to  perform  their  assigned  functions?  (Alternate 
location  for  5.2-e) 

_ 6.2:  Does  the  range  have  a  formal  training  program,  and  if  so,  are  records  kept  up  to  date? 

Is  the  training  offered  sufficient  to  ensure  or  maintain  technical  competence? 

_ 6.3:  Are  the  training  records  available  to  range  management? 


These  records  do  not  have  to  reside  in  the  range  book  but  must  be 
identified  by  location. 


_ On-site  Review:  Ask  to  locate  the  personnel  training  records  of  6.3. 

SECTION  7  -  ACCOMMODATION  AND  ENVIRONMENT  (PRIORITY  3) 

This  section  should  document  the  range  environment  as  well  as  any  and  all  parameters  of  the 
range  environment  that  may  affect  the  outcome  of  measurements  and  thereby  should  be 
monitored.  Some  environmental  variables  that  could  affect  radar  cross  section  measurements 
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include:  electromagnetic  interference,  temperature  variations,  mechanical  vibrations,  and 
seasonal  variations  in  vegetation,  sea  state,  moisture  levels,  winds,  precipitation,  inversions,  etc. 
(Depending  on  the  complexity  of  the  range,  this  section  may  or  may  not  be  significant.) 


Evaluation  Criteria: 

N/A  7.1:  (Deleted)  Same  criteria  is  found  in  section  10-7d. 

_ 7.2:  Are  the  nominal  measurement  environmental  factors  specified  for  acceptable 

operations  (e.g.,  allowable  wind  speeds,  temperature  variations,  etc.)? 

_ 7.3:  Are  appropriate  environmental  parameters  recorded  as  a  part  of  the  test  process? 

_ On-site  Review:  Are  the  environmental  factors  identified  being  recorded  in  a  log  or 

with  the  data?  For  environmental  measurements  being  recorded,  review  such  data  and 
operations  with  range  technicians. 

_ On-site  Review:  Is  there  any  measurement  data  showing  system  stability  or  repeatability 

versus  stated  control  values  for  environmental  variations?  (In  other  words,  are  the  factors  in 
7.2  derived  based  on  data  and  if  so,  what  data?) 


SECTION  8  -  EQUIPMENT  AND  REFERENCE  MATERIALS  (PRIORITY  2) 

Configuration  control  is  essential  for  maintaining  a  repeatable,  quality  RCS  system.  Therefore, 
this  section  of  the  range  book  may  be  one  of  the  most  significant.  Here  the  range  should  identify 
all  applicable  electro-mechanical  equipment  that  makes  up  the  RCS  range.  References  to 
appropriate  equipment  manuals  should  be  given  in  sufficient  detail  so  that  a  qualified  technician 
can  trace  the  major  subsystems  of  the  range.  Maintenance  and  calibration  histories  should  be 
kept  for  critical  components,  and  complete  system  configuration  should  be  tracked.  The 
procedures  for  identifying  equipment,  maintenance,  and  configuration  control  should  be 
documented  here.  Personnel  responsible  for  following  these  procedures  should  be  clearly 
identified.  For  electronic  (digital/intra-net)  range  books,  it  is  not  essential  to  include  an 
electronic  version  of  critical  range  equipment  in  this  section.  However,  the  titles  and  exact 
locations  of  externally  referenced  documents  must  be  included  in  this  section. 


The  numbering/lettering  sequence  below  follows  Z540. 


Evaluation  Criteria: 

_ 8.1:  Does  the  range  have  a  list,  data  base,  or  other  log  or  configuration  control  method  for 

identifying  and  tracking  the  main  range  equipment  systems  and  subsystems? 

_ 8.2a:  For  any  range  equipment  identified  in  8.1,  is  a  maintenance  schedule  identified  for 

periodic  or  preventive  maintenance? 

_ 8.2b:  Is  there  any  evidence  the  equipment  maintenance  schedules  are  being  followed 

(i.e.,  last  “check-on”  date,  maintenance  logs,  etc.)? 

_ 8.2c:  For  major  range  systems  or  subsystems,  are  maintenance  procedures  clearly 

identified  either  in  a  reference  manual  or  a  printable  procedure? 
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8.3a:  Is  any  individual  equipment  in  the  range  system  identified  as  to  whether  its  internal 
calibration  status  affects  the  level  of  range  RCS  calibrations?  (This  requirement  is 
N/A  if  end-to-end  system  calibration  is  performed  as  a  routine  practice  at  the  range.) 
8.3b:  For  any  equipment  identified  in  8.3-a,  is  the  calibration  status  and  interval  of  said 

equipment  marked  or  displayed  on  the  equipment?  (If  8.3a  is  N/A,  this  section  is  N/A.) 
8.4a:  Is  there  an  appropriate  reference  manual  for  each  piece  of  range  equipment  identified 
in  8.1?  (This  requirement  may  be  satisfied  with  a  data  base  or  listing  cross-referencing 
the  equipment  with  its  manual  and  its  exact  physical  location  when  not  in  use.) 

8.4b:  Does  the  document  and  equipment  log  show  the  name  and  current  location  of  the 
equipment? 

8.4c:  Does  the  document(s)  show  the  maintenance  history  of  these  pieces  of  equipment? 

8.4d:  Does  the  document  show  a  history  of  damage,  malfunction,  modification,  or  repair  of 
the  items? 

8.4e:  If  automated  testing  is  used  to  verify  subsystem  performance  (e.g.,  loop  test  modes, 
automated  power  supply  monitoring,  etc.),  are  non-compliant  loop  tests  documented 
and  corrected  (or  noted)  prior  to  system  use? 

On-site  Review:  Select,  at  random,  five  pieces  of  equipment  from  the  log  of  8.1  above. 

Can  the  pieces  of  equipment  be  located? 

On-site  Review:  Ask  to  see  the  maintenance  procedures  (if  applicable)  for  those  pieces 
of  equipment.  Are  they  up  to  date? 

On-site  Review:  Ask  to  see  the  reference  manuals  for  the  five  pieces  of  equipment 
selected.  Are  the  manuals  readily  available?  Were  those  manuals  found  in  the 
locations  stipulated  in  the  range  book? 

On-site  Review:  For  the  five  pieces  of  equipment  selected,  is  there  a  historical  log  kept 
on  damage,  repair,  or  recalibration  (if  appropriate)? 


SECTION  9  -  MEASUREMENT  TRACEABILITY  AND  CALIBRATION  (PRIORITY  1) 

This  section  should  complement  section  8  as  it  specifically  relates  to  primary  RCS  calibration. 
The  range  will  document  its  calibration  program  that  ensures  all  equipment  critical  to  RCS 
measurement  calibrations  operates  within  certified  performance  limits.  Equipment  and  system 
calibration  intervals  should  be  clearly  established.  Whenever  possible,  traceability  of  system 
components  and  of  total  system  calibration  to  a  national  standard  should  be  referenced.  The 
system  configuration  used  in  RCS  calibrations  should  be  tracked  using  the  method  documented 
in  section  8.  The  system  configuration  used  in  interrange  comparisons  should  be  clearly 
documented  together  with  the  results  of  calibration  comparisons.  System  components  tracked 
under  section  8  need  only  be  referenced  here. 


NOTE  ^  Z540  section  9.5  is  found  in  range  book  section  11. 

I  '  '  .  W 


n-12 


Evaluation  Criteria: 


9.1:  Does  the  range  have  an  established  and  documented  program  to  ensure  that  its 
calibration  program  supports  the  stated  calibration  performance  of  its  RCS 
measurement  system(s)? 

9.2:  Is  the  RCS  range  calibration  process  traceable  to  a  published  (AMTA,  IEEE,  etc.) 
calibration  artifact  standard  or  other  national  standard  (when  developed)? 


)  NOTE 


Under  normal  circumstances,  RCS  measurement  systems  are  calibrated  as 
a  total  “end-to-end”  system.  In  special  RCS  test  cases  where  specific  end- 
to-end  calibration  measurements  are  not  the  norm,  sections  9.3,  9.4,  and  9.6 
must  be  addressed  for  these  cases. 


9.3:  Does  the  range  measurement  and  calibration  program  include  a  top-level  schedule  for 
calibration  intervals  for  each  piece  of  test  equipment  used  to  assess  system 
performance? 

9.4:  Is  there  documentary  evidence  the  top-level  schedule  (9.3)  is  being  rigorously 
followed? 

9.6:  Does  the  range  have  a  documented  program  to  monitor  test  equipment  and/or 
calibration  artifacts  between  scheduled  calibrations  and  verifications? 

9.7:  When  the  range  participates  in  interrange  comparisons  (see  5.6),  is  the  range  RCS 
system  configuration  employed  in  those  comparisons  clearly  documented? 

On-site  Review:  For  the  primary  calibration  artifacts,  are  sample  “acceptable” 

calibrations  or  verification  records  available  within  the  range  book? 


SECTION  10  -  MEASUREMENT  AND  CALIBRATION  PROCEDURES  (PRIORITY  1) 

Both  measurement  and  calibration  procedures  should  be  fully  documented.  Separate 
documentation  should  be  available  for  wideband  calibration  and  measurement  procedures, 
monostatic  and  bistatic  (multistatic)  procedures,  and  polarimetric  (reciprocal  or  nonreciprocal) 
calibration  procedures.  Procedures  for  deriving  or  assigning  an  RCS  value  to  a  primary  target 
through  the  use  of  “transfer”  calibration  standards  should  be  documented  in  this  section. 
Calibration  and  measurement  results  to  support  the  validity  of  procedures  used  should  be 
displayed  or  referenced.  The  validity  of  such  results  should  be  supported  by  stated  uncertainty 
bounds  obtained  through  well-defined  uncertainty  procedures  appropriately  referenced  here  or  in 
Section  19.  Calibration  intervals  should  be  clearly  established. 


The  numbering/lettering  sequence  below  follows  Z540. 


NOTE 


/ 


Evaluation  Criteria: 

_ 10.1a:  Does  the  range  have  documented  system-wide  calibration  procedures? 
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10.1b:  Does  the  range  have  documented  system- wide  measurement  procedures  customized 
to  specific  target  mounts  used  by  the  range,  including  pylon  rotators  (if  applicable), 
foam  columns  (if  applicable),  air  columns  (if  applicable),  strings  (if  applicable),  or 
other  mounting  techniques? 

10.2a:  Are  separate  calibration  and  measurement  procedures  for  each  major  measurement 
system  or  for  each  type  of  measurement  conducted  available  (e.g.,  wideband, 
monostatic,  bistatic,  etc.)? 

10.2b:  Are  the  calibration  measurement  procedures  consistent  with  the  accuracy  required 
for  each  type  of  calibration  measurement  (as  defined  in  the  uncertainty  analysis  in 
section  19)? 

10.3:  Are  the  procedures  for  deriving  or  assigning  RCS  values  for  a  measured  target 
through  the  use  of  transfer  standards  documented? 

10.4:  Are  system- wide  calibrations  scheduled  on  an  appropriate  cycle  for  the  measurement 
system?  Is  this  cycle  defined?  Is  there  evidence  they  are  being  performed  when 
scheduled?  Are  the  results  of  system- wide  calibrations  included  or  referenced  in  this 
section?  Are  these  calibrations  provided  to  the  customer  as  part  of  their  normal  data 
report? 

10.5:  If  sampling  is  included  as  a  part  of  calibration  or  measurements,  is  documentation 
available  that  describes  the  procedures  and  applied  statistical  techniques  employed 
(detailed  data  processing  implementations  may  be  provided/discussed  in  section  18, 
if  applicable)? 

10.6:  Do  the  results  of  acceptable  system-wide  calibrations  support  the  uncertainty  bounds 
established  in  section  19? 


NOTE 


Computers  are  currently  an  integral  part  of  every  RCS  range.  Since  computers 


^ are  used  to  record,  store,  retrieve,  and  process  information,  certain  processes, 
*-x  ■  -  -  *  procedures,  and  documentation  need  to  be  made  available  to  the  operator. 

Normal  user  manuals  are  crucial,  and  on-line  “help”  features  of  commercial 
off-the-shelf  (COTS)  software  will  fulfill  a  large  portion  of  the  requirements  to  follow. 
Specialized  software  written  exclusively  for  range  data  acquisition  and  processing  requires 
equally  thorough  documentation. 


10.7a:  For  (any)  computer  software  used  to  acquire  and/or  store  the  raw  radar  data,  is 

there  an  available  users  manual(s),  either  as  automated  help  files  or  as  a  separate 
document(s),  which  describes  the  correct  operation  and  use  of  the  software?  If  the 
manual  is  not  embedded  in  the  range  book,  is  the  physical  location  of  the 
documentation  clearly  identified? 

10.7b  1:  For  computer  software  used  to  store  (locally  or  on  a  network)  or  transmit  from 

one  network  location  to  another,  are  appropriate  users  manuals  and/or  COTS  help 
files  available? 

10.7b2:  Is  there  a  data  base  or  other  appropriate  tracking  list  of  the  computer  program(s) 

that  are  used  in  the  end-to-end  data  acquisition,  transmission,  and  processing  steps? 

10.7b3:  For  computer  software  used  to  process  the  raw  data  files  into  final  data  products, 
are  appropriate  user  manuals  and/or  COTS  help  files  available?  (This  requirement 
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may  be  satisfied  one  of  two  ways.  If  specifically  written  users  manuals  have 
detailed  specific  examples,  these  can  be  used.  If  such  specific  examples  do  not 
exist,  the  requirement  can  be  made  by  appending  sample  “step-by-step”  processes 
that,  if  followed,  produce  the  final  data  products.) 

10.7c  1:  Is  there  an  assigned  computer  support  and/or  network  administrator  available  to 
maintain  computer  equipment  used  in  the  range  data  acquisition  and  processing? 

10.7c2:  Is  there  a  diagram  of  the  local  area  network  within  the  range,  if  apphcable? 

10.7d:  Are  critical  environmental  and/or  operating  procedures  monitored  to  maintain  the 
stated  calibration  accuracy  and/or  stay  within  the  published  calibration 
uncertainty  of  section  19? 

10.7e:  Are  there  procedures  that  ensure  security  of  the  data  to  prevent  the  unauthorized  or 
inadvertent  access,  manipulation,  or  destruction  of  computer  data  or  records? 


/r  — '  An  approved  network  automated  information  system  (AIS)  or  standard 
practices  and  procedures  (SPP)  plan  will  satisfy  this  requirement.  If  the 
AIS/SPP  is  not  electronically  available  in  the  range  book,  its  physical 
location  must  be  precisely  identified. 


On-site  Review:  Request  the  range  manager  perform  a  system-wide  calibration.  Verify  that 
the  documented  procedures  are  followed  by  range  technicians. 

On-site  Review:  Compare  the  results  of  the  system-wide  calibration  with  the  historical 
records  found  in  sections  12,  13,  and/or  19.  Are  the  results  within  the  stated  calibration 
uncertainty  limits? 

On-site  Review:  Ask  a  technician  if  the  observed  documented  procedures  are  up  to  date. 
Ask  when  the  procedures  were  last  reviewed  for  completeness  and  accuracy. 


SECTION  11  -  HANDLING  OF  CALIBRATION  ITEMS  (PRIORITY  1) 

This  section  should  summarize  how  primary  calibration  items  are  protected  during  storage, 

handling,  and  use.  Mechanical  tolerance  certificates  or  techniques  for  verifying  tolerances  of 

primary  standards  are  appropriately  placed  in  this  section. 

Evaluation  Criteria: 

_ 11.1:  Does  the  range  have  a  system  for  uniquely  identifying  primary  calibration  artifacts? 

_ 11.2:  Does  the  range  have  a  system  for  assessing  the  mechanical  tolerances  and  electrical 

conductivity  of  the  primary  calibration  artifacts  upon  initial  receipt?  Are  visual 
inspections  performed  on  the  calibration  artifacts  as  part  of  the  calibration  process? 
Is  there  a  stated  procedure  for  reacting  to  artifacts  found  or  suspected  to  be  damaged 
or  out  of  tolerance?  (  A  mechanical  test  and/or  measurement  certificate  will  satisfy 
the  initial  receipt  requirement.) 

_ 11.3:  Does  the  range  have  documented  procedures  and  appropriate  facilities  to  avoid 

deterioration  or  damage  to  calibration  artifacts  during  storage,  handling,  preparation, 
and  calibration? 
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) NOTE  / 


11.3  also  satisfies  Z540  section  9.5 


1 1 .4:  Does  the  range  book  identify  the  exact  location  of  the  calibration  artifact  standard 
when  not  in  use? 


/The  biggest  threat  posed  to  a  standard  RCS  artifact  device  (sphere,  cylinder, 
etc.)  is  physical  damage.  Most  calibration  artifact  devices  are  fabricated  from 
r'  aluminum,  which  generally  show  damage  (dents,  scratches,  etc.)  fairly  easily. 
Therefore,  it  makes  sense  to  store  the  calibration  artifacts  in  well  identified, 
padded  boxes  or  other  suitable  storage  devices. 


11.5:  Does  the  range  take  reasonable  and  appropriate  measures  to  protect  the  calibration 
artifact(s)  when  not  in  use?  Are  there  documented  procedures  for  reporting 
suspected  damage  to  a  calibration  artifact(s)?  If  the  artifacts  are  repaired  or  replaced, 
are  the  procedures  of  1 1 .2  followed  for  the  (repaired  or  replaced)  unit? 

11.6:  If  the  primary  calibration  artifacts  employed  by  the  range  are  used  in  any  capacity 
other  than  to  ensure  the  performance  of  the  RCS  measurement  system(s),  do  such 
uses  invalidate  or  degrade  the  calibration  artifacts? 

On-site  Review:  Perform  a  visual  inspection  of  the  range’s  calibration  artifacts  to  determine 
whether  the  conditions,  procedures,  and  facilities  assessed  in  11.1  through  11.5  are  as  stated 
in  the  range  book. 


SECTION  12  -  RECORD  KEEPING  (PRIORITY  3) 

This  section  describes  the  range’s  procedures  for  creating  and  maintaining  records  of  all  aspects 
of  range  operations.  Procedures  for  maintaining  calibration  records,  measurement  records, 
system  configuration,  etc.,  are  especially  important. 

Evaluation  Criteria: 

_ 12.1:  Does  the  range  have  well-documented  procedures  for  creating  and  maintaining 

records  of  range  operations,  to  include  records  of  calibration,  measurements,  and 
system  configuration? 

_ 12.2:  Are  these  records  indexed  or  stored  so  they  are  easily  located  and  accessible  to  those 

with  a  need  to  review  them? 
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)  NOTE 


This  section  may  be  evaluated  in  conjunction  with  the  examination  of 
the  records  required  by  other  sections,  such  as  13,  14,  15,  16,  17,  etc. 


SECTION  13  -  CERTIFICATES  AND  REPORTS  (PRIORITY  2) 

This  section  should  define  the  standard  reporting  formats  used  by  the  ranges  when  reporting  RCS 
measurements  to  customers.  While  any  report  can  be  specialized  to  the  requirements  of  the 
customer,  this  section  should  provide  the  baseline  information  that  will  always  be  available  in  a 
test  report.  A  range  is  allowed  to  use  its  own  format,  provided  that  the  minimum  information 
required  by  ANSI  Z540-1  is  included. 

Evaluation  Criteria: 

_ 13.1:  Does  the  range  book  include  a  description  of  and  purpose  for  each  type  of  certificate 

or  report  employed  or  delivered  by  the  range  (e.g.,  measurement  data  and  test 
summary  reports)? 

_ 13.2a:  Does  each  certificate  or  report  include  a  title  (e.g.,  Measurement  Report);  name  and 

address  of  the  facility;  points  of  contacts;  phone/fax  numbers;  location(s)  of  the 
test(s);  name,  address,  and  points  of  contact  of  the  customer;  description  and 
unambiguous  identification  of  the  measured  item  (including  its  configuration,  as 
appropriate);  characterization  and  condition  of  the  measured  item;  date(s)  and 
times(s)  of  the  measurement(s);  identification  of  the  calibration  procedure(s)  used 
or  unambiguous  description  of  non-standard  procedures  employed;  reference  to 
sampling  and/or  data  processing  procedures,  where  relevant;  any  deviations  from, 
additions  to,  or  exclusions  from  documented  operating  procedures;  and  other 
information  relevant  to  the  measurement(s),  such  as  environmental  conditions  or 
failures/anomalies  encountered  during  the  measurement(s)? 

_ 13.2b:  Does  each  certificate  or  report  include  measurements,  examinations,  and  derived 

results  supported  by  tables,  graphs,  sketches,  and  photographs  (and/or  digital 
representations  of  such),  as  appropriate  for  the  measurements  conducted? 

_ 13.2c:  Does  each  certificate  or  report  include  a  statement  of  the  estimated  uncertainty  of 

the  primary  calibration  results  and  the  traceability  of  the  measurements? 

_ 13. 2d:  Does  each  certificate  or  report  include  a  signature  and  title  of  the  person(s) 

accepting  responsibility  for  the  content  of  the  certificate  or  report,  and  the  date  of 
issuance? 

_ 13.2e:  Does  each  certificate  or  report  include  a  statement  regarding  any  limitations  on  the 

use  or  interpretation  of  the  measurement  data? 

_ 13.3:  Are  portions  of  the  measurement  data  products  performed  by  subcontractors?  If  so, 

are  such  portions  clearly  identified?  (This  will  probably  be  N/A  for  most  ranges.) 

_ 13.4:  Is  there  a  procedure  for  amending  a  report?  Is  there  a  procedure  for  notifying 

customers  promptly,  in  writing,  of  any  event  such  as  discovery  of  defective 
calibration  equipment  that  may  cast  doubt  on  the  validity  of  the  results  provided  in 
any  given  report  or  certificate,  including  the  magnitude  of  the  errors  that  may  exist? 
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13.6:  Is  there  a  documented  procedure  for  transmitting  reports  or  certificates  to  customers 
or  third  party  recipients  such  that  confidentiality  is  preserved? 


A  hand  receipt,  computer  receipt,  registered  mail  receipt,  or  other 
appropriate  tracking  system  may  be  used  to  fulfill  this  requirement. 


On-site  Review:  None  for  this  section. 


SECTION  14  -  SUBCONTRACTING  RECORDS  (PRIORITY  3) 

This  section  details  any  aspects  of  the  RCS  calibration  process  that  are  subcontracted.  Any 
subcontracted  work  must  satisfy  the  ANSI  Z540-1  standard.  (This  section  may  not  be  applicable 
for  most  ranges.)  The  main  intent  of  this  section  is  to  address  calibration  activities  sent  to 
locations  or  personnel  that  are  not  within  the  range  facility  (on-site).  When  on-site  contractors 
share  complete  responsibility  for  operating  a  range  facility,  as  is  often  the  case,  we  refer  to  this 
situation  as  an  integrated  product  team.  For  example,  AFRL  hires  an  on-site  contractor  to  assist 
in  range  operations.  This  is  not  considered  subcontracting.  However,  if  a  primary  range  relies 
on  an  off-site  secondary  RCS  range  for  calibration  data  or  other  measurements  applicable  to  the 
primary  range,  this  situation  would  be  treated  as  a  subcontracted  measurement,  and  section  14 
would  apply.  For  most  ranges,  these  criteria  probably  will  not  apply. 

Evaluation  Criteria: 

_ 14.1:  If  the  range  employs  any  subcontracting  in  the  performance  of  RCS  measurements,  is 

there  a  documented  procedure  or  policy  that  stipulates  and  verifies  that  such 
Does  the  range 

maintain  records  of  the  work  accomplished  by  subcontractors? 

_ 14.2:  If  the  range  employs  a  contractor  to  perform  measurements,  does  the  contract  or  other 

operational  document  bind  the  contractor  to  the  procedures  defined  by  the  range 
book? 

_ On-site  Review:  Ask  the  range  manager  whether  subcontractors/contractors  support 

or  perform  RCS  measurements.  If  applicable,  review  the  contract(s)  to  verify  compliance 
with  the  ANSI-Z540  requirements. 


SECTION  15  -  OUTSIDE  SUPPORT  SERVICES  AND  SUPPUIERS  (PRIORITY  3) 

This  section  should  document  any  services  or  suppliers  used  to  produce  calibrated  RCS  data. 
This  section  complements  section  14.  For  instance,  if  a  range  purchases  an  8-inch  calibration 
sphere  from  a  vendor  for  primary  calibration,  this  section  would  document  the  needed  tolerances 
of  such  a  standard,  as  well  as  the  delivered  tolerance  of  the  calibration  item.  As  an  additional 
example,  if  radar  absorbing  material  (RAM)  is  used  on  a  calibrated  target  measurement  and 
absolute  RCS  data  is  reported,  it  would  be  appropriate  to  create  records  of  where  the  RAM  was 
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purchased,  what  characteristics  were  expected  and  delivered,  and  what,  if  any,  repeatability 

would  result  from  reusing  the  same  material  samples  on  the  same  target  location. 

Evaluation  Criteria: 

_ 15.1:  If  the  range  uses  outside  services  or  suppliers  that  might  impact  the  quality  and 

accuracy  of  RCS  measurements,  are  there  documented  policies  that  ensure  the  quality 
of  the  calibrations  and  measurement  products? 

_ 15.2:  Does  the  range  book  identify  outside  services  or  suppliers,  or  is  there  a  definitive 

statement  that  no  outside  suppliers  or  services  are  employed  in  the  calibration  and 
measurement  functions  of  the  range? 

_ 15.3:  Where  applicable,  are  there  procedures  for  maintaining  records  associated  with 

outside  services  or  suppliers? 

_ On-site  Review:  If  outside  suppliers  or  services  are  used  at  the  range  under  review, 

ask  to  see  the  associated  records. 


SECTION  16  -  COMPLAINTS  (PRIORITY  3) 

This  section  should  document  the  formal  complaint  procedure  used  to  resolve  disputes  between  the 

range  and  a  customer.  The  complaint  record  keeping  system  also  should  be  defined  or  referenced 

here. 

Evaluation  Criteria: 

_ 16.1:  Does  the  range  have  documented  procedures  for  the  resolution  of  customer  complaints? 

_ 16.2:  Are  records  maintained  of  all  complaints  and  of  the  actions  taken  by  the  range  to 

resolve  the  complaints? 

_ 16.3:  If  a  customer  complaint  raises  a  concern  regarding  the  range’s  compliance  with  range 

policies  or  procedures,  or  with  the  requirements  of  this  standard,  does  the  range  ensure 
that  these  complaints  are  reviewed  and  acted  upon  by  a  level  of  authority  at  least  as 
high  as  the  signatory  identified  in  section  1?  Is  this  individual  able  to  effect  the 
required  changes  in  range  policy  and  procedures,  if  needed? 

_ 16.4:  If  the  customer  complaint  raises  a  concern  regarding  the  quality  of  the  range’s 

calibrations,  does  the  range  ensure  that  these  complaints  are  reviewed  and  acted  upon 
by  a  level  of  authority  at  least  as  high  as  the  signatory  identified  in  Section  1?  Is  this 
individual  able  to  effect  the  required  changes  in  range  policy  and  procedures,  if 
needed? 

_ On-site  Review:  Examine  records  of  a  random  complaint  (if  applicable)  to  ensure  that  the 

complaints  were  reviewed  and  that  appropriate  action  was  taken  and  recorded. 

_ On-site  Review:  Ask  range  personnel  if  they  are  familiar  with  the  range’s  complaint 

resolution  process. 

_ On-site  Review:  Ask  the  range  quality  manager  if  and  how  he/she  is  involved  in  the 

complaint  resolution  process.  Does  the  range  quality  manager  review  all  customer 
complaints  for  indications  that  current  policies  and  procedures  are  in  need  of  review? 
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SECTION  17  -  INTERRANGE  COMPARISON  PROGRAMS  (PRIORITY  1) 


This  section  shall  include  the  organization’s  policy  regarding  interrange  comparisons.  Records 
of  participation  in  calibration  interrange  comparisons,  the  results  obtained,  and  conclusions 
drawn  from  such  studies  are  to  be  presented  or  referenced  here.  If  the  information  is  already 
included  in  section  10,  a  reference  to  section  10  is  adequate.  It  is  expected  that  interrange 
comparisons  will  be  used  regularly  to  improve  measurement  and  calibration  quality  at  the  range. 
An  interrange  comparison  schedule  should  be  provided. 


/NOTE 

/ 


Participation  in  a  national  interrange  comparison  study  is  required  for 
certification. 


Evaluation  Criteria: 

_ 17.1:  Does  the  range  have  a  documented  policy  on  interrange  comparisons? 

_ 17.2:  Does  the  range  participate  in  an  interrange  calibration  comparison  program  at  the 

national  level? 

_ 17.3:  Are  the  results  of  the  interrange  calibration  comparisons  of  section  17.2  available  for 

customer  review? 

_ 17.4:  Are  the  results  of  the  interrange  comparisons  used  to  evaluate  and  improve  range 

operations? 

_ On-site  Review:  Review  the  range’s  documented  interrange  comparison  policy.  Is  the 

available  data  current?  Does  the  range  have  a  schedule  for  participation  in  interrange 
comparisons? 

_ On-site  Review:  Ask  the  range  technical  manager  how  the  results  of  the  interrange 

comparisons  are  applied  to  improving  the  quality  of  range  measurements. 


SECTION  18  -  DATA  PROCESSING  PROCEDURES  (PRIORITY  1) 

Important  details  of  processing  measurement  and  calibration  data  should  be  documented  here. 
Sample  results  of  processing  should  be  included.  Standard  data  processing  procedures  should  be 
referenced,  and  the  validity  of  innovative  processing  should  be  documented.  To  evaluate  the 
validity  of  its  data  processing  procedures,  the  range  should  use  a  standard  data  set  acceptable  to 
the  RCS  industry.  This  standard  data  set  should  be  used  by  the  range  to  perform  interrange 
comparison  of  its  processing  algorithms  with  those  used  by  other  ranges.  This  section  should 
strongly  support  and  complement  the  material  in  section  17  above. 

Evaluation  Criteria: 

_ 18.1:  Does  the  range  have  all  primary  RCS  and  image-based  data  processing  procedures 

documented? 
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18.2:  Have  the  data  processing  modules  been  validated  with  a  standard  data  set  to  ensure 
the  correctness  of  the  algorithms  employed? 


NOTE 


A  generic  set  of  C-29  model  RCS  data  is  available,  if  desired,  to  assist 
a  range  in  assessing  this  requirement. 


_ 18.3:  Are  the  results  of  data  processing  compared  against  known  quantities  or  standards  to 

verify  accuracy  of  the  measurement  and  data  processing  process? 

_ On-site  Review:  Is  there  a  procedure  to  collect,  determine,  and  validate  all  customer  data 

processing  requirements,  to  include  such  key  items  as  specific  formats,  parameters,  and 
quantities?  Is  there  a  procedure  for  translating  user  requirements  into  the  specific 
procedures  to  collect  the  data  identified  by  the  customer? 

_ On-site  Review:  Are  procedures  in  place  to  coordinate  target  readiness  and  state  with 

data  collectors  and  data  processors  so  that  the  target  configuration  is  captured  throughout  the 
data  collection  and  processing  chain? 

_ On-site  Review:  Are  procedures  in  place  for  validating  the  data  in  (near)  real-time  to 

ensure  the  proper  data  is  being  collected? 

_ On-site  Review:  Ask  a  range  technician  to  process  a  file  from  a  standard  data  set  and 

compare  the  results  to  those  known  to  be  correct. 

_ On-site  Review:  Ask  the  range  technical  manager  how  new  or  improved  data  processing 

algorithms  are  validated. 

_ On-site  Review:  Review  the  results  of  any  interrange  data  processing  comparisons.  Are 

these  results  available  to  range  customers? 

SECTION  19  -  RANGE-SPECIFIC  CALIBRATION  UNCERTAINTY  ANALYSIS 
(PRIORITY  1) 

Policies  and  procedures  for  establishing  range  uncertainties  need  to  be  completely  documented  or 
referenced  here.  A  sample  uncertainty  table  together  with  system  parameters  or  other  adequate 
uncertainty  methods  should  be  linked  or  available  in  this  section.  One  approach  to  satisfy  this 
requirement  is  the  “Report  of  Measurement”  described  in  [15].  Any  system  parameters  should  be 
displayed  here.  Any  scientifically-based  uncertainty  analysis  is  acceptable,  providing  all  appropriate 
assumptions  and/or  exclusions  regarding  the  parameters  comprising  the  analysis  are  clearly 
documented. 

Evaluation  Criteria: 

_ 19.1:  Does  the  range  have  a  policy  for  the  development  and  use  of  a  calibration  uncertainty 

analysis?  (Alternate  location  for  section  5.2u.) 

_ 19.2:  For  the  primary  calibration  standard(s)  used  by  the  range,  is  there  a  nominal  RCS 

measurement  with  estimated  uncertainties  in  either  graphical  or  tabular  form  for  that 
standard? 
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) NOTE  / 

.  W 

A  NIST  calibration  certificate  may  be  used  to  fulfill  this  requirement,  but 
other  formats  containing  similar  information  are  acceptable. 

_ 19.3:  Is  the  calibration  uncertainty  shown  in  19.2  consistent  with  stated  calibration 

capabilities  of  the  range  in  other  range  book  sections? 

) NOTE ^ 

-  ,  w 

If  the  range’s  stated  calibration  accuracy  is  less  than  the  accuracy  shown  in 
this  nominal  measurement,  those  capabilities  must  be  changed  to  match 
those  obtained  in  the  section  19.2  measurements. 

19.4:  Are  the  results  of  the  uncertainty  analysis  for  primary  calibration  reflected  in  the 
day-to-day  measure  of  calibration  quality  discussed  in  section  10?  In  other  words, 
are  typical  uncertainties  used  as  a  quality  check  for  daily  calibration  measurements? 
19.5:  Does  the  range  have  a  “Report  of  Measurement”  for  its  most  commonly  used 
calibration  artifact  stating  its  expected  uncertainty  under  nominal  operating 
conditions? 

On-site  Review:  Ask  the  radar  technicians  to  show  (in  tabular  or  graphical  form)  the 
predicted  uncertainty  of  the  primary  calibration  standard.  Can  the  technicians  find 
the  information? 

On-site  Review:  Do  the  technicians  use  the  primary  calibration  target  uncertainty  bars 
to  bracket  acceptable  (daily)  calibration  as  a  quality  check? 

On-site  Review:  Ask  the  following  question  and  assess  answer:  Are  the  actions  taken  by  the 
technicians  when  calibration  measurements  fall  outside  expectations  consistent  with  the 
stated  range  policy  on  exceptions  to  approved  calibration  process?  (See  sections  5  and  10.) 


SECTION  20  -  ONGOING  RESEARCH,  PLANNED  IMPROVEMENTS  (PRIORITY  2) 

The  purpose  of  this  section  is  to  briefly  summarize  the  three-year  look  ahead  regarding  any  range 
research  and/or  other  activities  designed  to  improve  range  data  quality,  efficiency,  repeatability, 
or  traceability.  A  list  of  desirable  research  areas  to  be  conducted  in  the  future  to  improve 
specific  and  known  deficiencies  on  the  range  or  in  the  RCS  industry  should  also  be  provided,  as 
well  as  a  single  top-level  roadmap.  Ongoing  plans  for  future  upgrades  of  equipment  should  be 
included.  Such  research  information  will  allow  customers  to  quickly  identify  and  evaluate 
ongoing  range  improvements  in  the  context  of  their  current  or  planned  use  of  the  range.  It  also 
may  offer  the  customer  an  opportunity  to  cost  share  or  jointly  sponsor  range  research  of  interest 
to  the  customer. 

Evaluation  Criteria: 


20-1:  Is  there  a  description  or  summary  of  a  three-year  look  ahead  into  future  plans, 
upgrades,  and  quality  or  range  improvements? 

20-2:  Is  there  a  roadmap  formatted  to  summarize  the  three- year  plan? 
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/NOTE 


The  roadmaps  may  be  longer  than  three  years  but  must,  at  a  minimum, 
look  ahead  three  years. 


On-site  Review:  As  a  result  of  the  review  committee  feedback,  is  there  a  summary  of 
topics  and  issues  that  will  be  worked  on  a  time-available  basis  over  the  succeeding 
three  years? 
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supported  it  even  when  issues  arose  that  threatened  its  success.  The  persistence  and 
integrity  of  this  dedicated  group  are  to  be  highly  commended. 

A  portion  of  this  project  was  supported  by  the  Air  Force  Information  Warfare 
Center  under  contract  F41621-97-D5200,  and  by  several  additional  contracts  and 
subcontracts  sponsored  by  the  Air  Force  Research  Laboratory  and  the  U.S.  Navy  Atlantic 
Test  Range. 
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